close
Skip to main content

r/pythoncoding


nbpipe: A Lightweight Workflow Runner for Jupyter Notebooks
nbpipe: A Lightweight Workflow Runner for Jupyter Notebooks

Hi r/Jupyter,

I've spent a good amount of time speaking with Jupyter users for my day-job, and I've noticed the same pattern over-and over. A workflow is split up across several notebooks, each creating a temp file that flows into the next notebook.

I have done this myself, and admittedly it's not the end of the world. But still annoying to open three notebooks, and run every cell.

With this in mind, I thought it would be nice to define workflows in a plain text file file, and be able to run them with one click. Think Github actions, but local, and less setup.

Enter nbpipe, a Jupyter extension to do just this. You create a YAML file with steps to run, and file validation (if you want). Then from Jupyter, you can run then workflow.

GitHub: https://github.com/ngafar/nbpipe

I would love any feedback or feature requests you may have.

upvotes comments

Have great products but nowhere to sell them? Get everything you need to start your online business in one place.
media poster


Pulumi Gcp And Python Up And Running

A faster, aria2 enabled pip wrapper
A faster, aria2 enabled pip wrapper

Installing large packages like torch or tensorflow has always been a pain in the ass for me. Although i have optimal internet connection, the download speeds i get for these packages is abysmal.

I then started to download the whl files through download managers then install them manually, and so to automate that workflow i created Zipip. It's just pip but with slight modifications to use aria2 for the downloads. And i dont mean it in any fancy sense, it literally is just pip in all other regards. And so likewise it is a drop in alternative.

Source code : https://github.com/STRTSNM/zipip

PyPI : https://pypi.org/project/zipip/

PS : I know pip is supposed to have no other dependencies, but the slow downloads just kill me, and i wanted to share what i thought of. Do let me know what you people think. Thanks in advance.