Azure Cloud is a popular work environment for many data scientists, yet many features remain poorly documented. This note shows how to monitor Azure experiments in a more handy and detailed way than through web or cl interface. The trick is to create a dashborad of experiments and their respective runs, up to a desired …
Category archives: coding
Repo Passwords in Poetry
Poetry, a popular Python package manager, prefers to use keyring to manage passwords for private code repositories. Storing passwords in plain text is a secondary option, but may be needed in case of either issues in poetry itself or with keyring configuration (may not be properly installed, be locked etc). To disable the use of …
Marking Python Tests as Optional
Often code tests are to be run on special demand, rather than in a CI/CD manner: for instance, they may be slow or work only in a local mode with protected data. This note shows how to declare code tests optional in pytest, the leading testing framework for Python. The article is inspired by the …
Debug CI/CD with SSH
What to do when CircleCI reports are not informative enough on errors? Debug the failing environment live with SSH!
Prototype in Jupyter on Multiple Kernels
For data scientists, it is a must to prototype in multiple virtual environments which isolate different (and often very divergent) sets of Python packages. This can be achieved by linking one Jupyter installation with multiple Python environments. Use the command <code>which jupyter</code> to show the Jupyter location and <jupyter kernelspec list> to show available kernels, …
Robust Azure ETLs with Python
Microsoft Azure faces criticism for being poorly explained, but remains a popular cloud computing platform for many companies. How data engineers can build robust extract-load-transform processes on top of it using Python
?
Demystifying TestDom
Recruiters nowadays use online timed tests when screening developers. I recently looked at Python & Algorithms Hard questions at TestDome. While the timing and hints seem to push towards implementing tricks from scratch, for the quality in long term it is better to structure the problem and use established solutions (divide & conquer). The battery …