1

In Azure Databricks I have I have a repo cloned which contains python files, not notebooks.

In Azure Data Factory I want to configure a step to run a Databricks Python file. However when I enter the /Repos/..../myfile.py (which works for Databricks Notebooks) it gives me the error " DBFS URI must starts with 'dbfs:'"

How can I reference a python file from a report which is not in dbfs?

enter image description here

NOTE I see a duplicate question here but the answer was just to wrap it in a Databricks Notebook - OK workaround but when I do it I get "No module named 'my_python_file'"

https://stackoverflow.com/questions/70096408/how-to-create-a-databricks-job-using-a-python-file-outside-of-dbfs

Brendan Hill
  • 301
  • 1
  • 4
  • 11

1 Answers1

1

For reference

After all the mucking around, we decided to build a Python package (wheel) which can be imported to Databricks as a library. This can then be installed with %pip install /dbfs/..../*.whl and the package is then accessible to notebooks.

Brendan Hill
  • 301
  • 1
  • 4
  • 11