python

7 tutorial series · 2 projects · 14 posts

Tutorial Series

Posts

pgmonkey: One Config File to Rule All Your PostgreSQL Connections

pgmonkey is a handy Python library that simplifies PostgreSQL connection management with just one YAML config file. It supports multiple connection types—like sync, async, and pooled—while handling caching and lifecycle tasks seamlessly. Ideal for projects with mixed connection needs, it saves developers from configuration chaos. Worth checking out!

Your Manager’s Spreadsheet Is Broken. Here’s the 3-Minute Fix.

Tired of dealing with messy spreadsheets that complicate your data workflow? Enter xlfilldown — the ultimate solution for data engineers seeking clean, queryable data. This powerful tool efficiently fills hierarchical data, ensuring context is preserved without memory overload. With a simple command, you can cleanse and streamline large Excel files, saving precious time while avoiding tedious manual fixes. Embrace hassle-free data management today!

The Secret Hierarchies of Excel: Why Managers Merge, and Engineers Cry

Excel may appear simple, but it conceals a daunting data challenge: the hierarchical spreadsheet. Common among analysts, it presents issues like merged cells and hidden hierarchies that frustrate data processing. Enter xlfilldown—a powerful solution that transforms chaotic Excel files into clean, structured datasets for seamless analysis. By bridging the gap between visual reports and data integrity, xlfilldown revolutionizes the ETL process for data professionals.

One Million Rows: Testing xlfilldown Performance and Constant-Memory Design

The content discusses the performance and reliability of the xlfilldown tool, which efficiently processes large Excel datasets without excessive memory usage. A stress test processing 1,000,000 rows demonstrated rapid speeds—13,800 rows/sec for Excel and 44,700 rows/sec for SQLite—while maintaining deterministic and auditable output. This enhances data integrity and workflow trust.

The State Of Python ‘setuptools’ ‘pyproject.toml’ Packaging In 2024

Pleasantly surprised when revisiting python packaging with setuptools and TOML in 2024.

Working With Git Submodules

Using Git Submodules with your python packages helps stop you from re-inventing the wheel. Make one wheel, and use it in every project. Let me show you how.

Reading & Writing To An SQLite3 Database Inside A Python Package

You might need to store data at the package level, take a look at the following package structure. I am going to show you how to create, read and write to a database at the package level. This means that when a user installs your package, you could store data in the python package. We’re...

Reading & Writing To Datafiles Inside A Package

There may be occasions where you want to store application data and configuration files outside of the reach of your users, and store it directly in the package files themselves.

Python Parsing Command Line Arguments With argparse

The python package ‘argparse’ is a highly versatile and customizable command line parser. I’m going to show you a handful of examples that should cover most of your use cases, if not you can reference the full ‘argparse’ documentation here. Come on – let’s take a look. Prerequisites Although you could apply the examples we...

Python Packages As Callable Console Commands

Wouldn't you like to be able to wrap up all of your cool code in to a package, and then call your code from the command line using a user friendly command name? Sure you do! Come on, let me show you how.

Bumper Python Package Installation Guide

There are a couple of different ways to install python packages. Come on, let's take a look at them.

Python Package Interactive Install / Editable Install With “pyproject.toml”

I've got a good one for you today. What a ride. It seems I have caught "setuptools" at a transitional moment. Installing python packages in editable / development mode with TOML "pyproject.toml" files is now possible - but a little rough around the edges for now. Come on - let's take a look.

Python Packaging – Using “setuptools”, “pyproject.toml”

You need to know how to package your python projects properly. Here I show you how to package, and submit your package to your very own pypi.org repository. I create the package "pkgexample" version 1.0.1 , your package name will vary if you follow along.

Google Service Account Authentication For Python API Calls: A Simple Example

This content provides a comprehensive guide on authenticating Google Cloud service accounts for Python API calls, detailing three methods: explicit file loading, Application Default Credentials, and manual token extraction. It emphasizes the importance of service accounts, API enablement, and maintaining security with key files, while offering practical coding examples and best practices.