dataflow_template
To get started:
Dynamically pull and run
from hamilton import dataflows, driver
# downloads into ~/.hamilton/dataflows and loads the module -- WARNING: ensure you know what code you're importing!
dataflow_template = dataflows.import_module("dataflow_template", "example_dataflow_template")
dr = (
driver.Builder()
.with_config({}) # replace with configuration as appropriate
.with_modules(dataflow_template)
.build()
)
# If you have sf-hamilton[visualization] installed, you can see the dataflow graph
# In a notebook this will show an image, else pass in arguments to save to a file
# dr.display_all_functions()
# Execute the dataflow, specifying what you want back. Will return a dictionary.
result = dr.execute(
[dataflow_template.CHANGE_ME, ...], # this specifies what you want back
inputs={...} # pass in inputs as appropriate
)
Use published library version
pip install sf-hamilton-contrib --upgrade # make sure you have the latest
from hamilton import dataflows, driver
# Make sure you've done - `pip install sf-hamilton-contrib --upgrade`
from hamilton.contrib.user.example_dataflow_template import dataflow_template
dr = (
driver.Builder()
.with_config({}) # replace with configuration as appropriate
.with_modules(dataflow_template)
.build()
)
# If you have sf-hamilton[visualization] installed, you can see the dataflow graph
# In a notebook this will show an image, else pass in arguments to save to a file
# dr.display_all_functions()
# Execute the dataflow, specifying what you want back. Will return a dictionary.
result = dr.execute(
[dataflow_template.CHANGE_ME, ...], # this specifies what you want back
inputs={...} # pass in inputs as appropriate
)
Modify for your needs
Now if you want to modify the dataflow, you can copy it to a new folder (renaming is possible), and modify it there.
dataflows.copy(dataflow_template, "path/to/save/to")
Purpose of this module
Template module to show what needs to be filled out.
Configuration Options
This module can be configured with the following options: [list options]
Limitations
Write limitations/assumptions/known issues here.
Source code
__init__.py
Requirements
# put non-hamilton requirements here