Jenkins CI/CD (11/11): Publish to PyPI Securely

Summary: You add the final stage to the helloci pipeline — publishing the built package to PyPI using twine. You create a scoped PyPI API token, store it as a Jenkins credential, and wire a “Publish to PyPI” stage that only fires on tagged builds. The result is a complete eleven-stage CI/CD pipeline that takes a Git tag all the way from checkout to a publicly installable package on PyPI, with credentials that never appear in source control or build logs.

Example Values Used in This Tutorial

Key	Value
Credential type	Secret Text
Credential ID	pypi-token
Twine command	twine upload dist/*
Token user	__token__
TestPyPI flag	--repository testpypi
Previous parts	Parts 1-10 completed

0. Prerequisites

• A working Jenkins controller at http://localhost:8080 (Part 1).
• The helloci Python package with unit tests, lint, integration tests, artifact archiving, workspace cleanup, concurrency control, and a tag-triggered Build Release stage in the pipeline (Parts 2-10).
• A PyPI account at pypi.org. If you do not have one, create a free account before continuing.
• A TestPyPI account at test.pypi.org (optional, but recommended for testing).
• The helloci package version set correctly in pyproject.toml — PyPI rejects duplicate version uploads.

Note: If you completed Part 10, your pipeline already builds wheels and sdists on tagged commits. This tutorial adds the publish step that uploads those artifacts to PyPI.

---

1. Why Secrets Management Matters

Your PyPI API token is a credential that grants permission to upload packages under your account. If it leaks, anyone can push malicious code to your package. Treating it casually leads to real damage.

Here are the ways credentials leak in CI/CD systems:

• Hardcoded in the Jenkinsfile. The token sits in source control, visible to anyone who can read the repository. Every fork, every clone, every backup carries the secret.
• Passed as a plain build parameter. Jenkins logs the parameter value in the build history, where it is visible in the UI.
• Echoed in a shell command. A debug echo or a verbose flag prints the token to the console output, which Jenkins stores permanently.
• Stored in an unencrypted file on the agent. Other builds on the same agent can read it.

Jenkins Credentials solves all four problems. Credentials are stored encrypted on the controller, injected into builds at runtime, and automatically masked in console output. The token never touches source control, never appears in build logs, and is only available inside the specific block where you request it.

---

2. Create a PyPI API Token

Go to pypi.org and log in. Navigate to Account Settings > API tokens and click Add API token.

Configure the token:

• Token name: jenkins-helloci (or any descriptive name).
• Scope: Select Scope: Project and choose helloci from the dropdown.

Warning: Do not create an account-wide token. Scoping the token to a single project limits the blast radius if the token is compromised. An attacker with a project-scoped token can only upload to helloci, not to every package in your account.

Click Create token. PyPI displays the token exactly once. It starts with pypi- and is a long string. Copy it immediately — you cannot view it again after leaving the page.

Tip: If you lose the token, you cannot retrieve it. Revoke the old one and create a new one. This is a feature, not a limitation — it prevents stale tokens from lingering.

---

3. Store the Token in Jenkins Credentials

Open Jenkins at http://localhost:8080. Navigate to Manage Jenkins > Credentials > (global) > Add Credentials.

Fill in the form:

• Kind: Secret text
• Scope: Global
• Secret: Paste the PyPI API token you copied in the previous step.
• ID: pypi-token
• Description: PyPI API token for helloci

Click Create.

The token is now stored encrypted in Jenkins. You reference it in the pipeline by the ID pypi-token — the actual token value never appears in your Jenkinsfile.

Verify the credential exists by going to Manage Jenkins > Credentials and confirming pypi-token appears in the list. The secret value is hidden — Jenkins shows only the ID and description.

---

4. Install Twine in the Pipeline

twine is the standard tool for uploading Python packages to PyPI. It handles authentication, TLS, and the PyPI upload API.

You do not need to install twine globally on the Jenkins agent. Install it inside the virtual environment, in the same stage where you use it. This keeps the dependency scoped to the publish step and ensures a consistent version across builds.

The install command runs inside the withCredentials block in the next section, so twine is only installed when the publish stage actually executes — that is, only on tagged builds.

---

5. Add the Publish to PyPI Stage

Add a new stage to the Jenkinsfile after the Build Release stage. Like Build Release, it only runs on tagged commits:

stage('Publish to PyPI') {
    when {
        buildingTag()
    }
    steps {
        withCredentials([string(credentialsId: 'pypi-token', variable: 'PYPI_TOKEN')]) {
            sh '.venv/bin/pip install twine'
            sh 'TWINE_USERNAME=__token__ TWINE_PASSWORD="$PYPI_TOKEN" .venv/bin/twine upload dist/*'
        }
    }
}Code language: Groovy (groovy)

There are several things happening in this stage:

• when { buildingTag() } ensures the stage only runs when Jenkins is building a Git tag, not on regular branch pushes. This matches the Build Release stage from Part 10.
• withCredentials pulls the pypi-token secret from Jenkins Credentials and exposes it as the environment variable PYPI_TOKEN inside the block.
• .venv/bin/pip install twine installs twine into the existing virtual environment.
• .venv/bin/twine upload dist/* uploads every file in the dist/ directory — both the wheel and the sdist that the Build Release stage created.
• TWINE_USERNAME=__token__ tells PyPI that you are authenticating with an API token, not a username/password. The literal string __token__ is the required username when using token-based authentication.
• TWINE_PASSWORD="$PYPI_TOKEN" passes the token via environment variable rather than a command-line flag. This avoids exposing the token in process listings (ps aux), where CLI arguments are visible to other users on the same machine.

---

6. How withCredentials Protects Your Token

The withCredentials block does three things:

• Injects the secret. The PYPI_TOKEN environment variable exists only inside the withCredentials block. Outside the block, the variable is undefined.
• Masks the value in logs. If the token value appears anywhere in the console output — in an error message, a debug trace, or a verbose flag — Jenkins replaces it with ****. This is automatic and requires no configuration.
• Limits the scope. The token is available only to the shell steps inside the block. Other stages, other builds, and other jobs on the same agent cannot access it.

This is why the twine upload command lives inside withCredentials rather than in a separate step. Moving it outside the block would mean $PYPI_TOKEN is empty, and the upload would fail with an authentication error.

Warning: Never add echo $PYPI_TOKEN or set -x inside the withCredentials block. While Jenkins masks known secret values in output, it is better to avoid printing credentials entirely. Defense in depth — do not rely on a single layer of protection.

---

7. The Complete Final Jenkinsfile

This is the full Jenkinsfile for the helloci project — the culmination of all eleven parts in this series. Replace your existing Jenkinsfile with this version:

pipeline {
    agent any
    options {
        timeout(time: 15, unit: 'MINUTES')
        disableConcurrentBuilds()
    }
    stages {
        stage('Setup Python') {
            steps {
                sh 'python3 -m venv .venv'
                sh '.venv/bin/pip install --upgrade pip'
            }
        }
        stage('Install Dependencies') {
            steps {
                sh '.venv/bin/pip install -e ".[test]"'
            }
        }
        stage('Lint') {
            steps {
                sh '.venv/bin/ruff check src/ tests/'
            }
        }
        stage('Unit Tests') {
            steps {
                sh 'mkdir -p results'
                sh '.venv/bin/pytest tests/test_greet.py --junitxml=results/junit.xml'
            }
        }
        stage('Integration Tests') {
            steps {
                retry(2) {
                    sh 'docker compose up -d'
                }
                sh 'docker compose exec -T postgres pg_isready -U testuser -d testdb --timeout=30'
                sh '.venv/bin/pytest tests/test_integration.py --junitxml=results/junit-integration.xml -v'
            }
        }
        stage('Build Release') {
            when {
                buildingTag()
            }
            steps {
                sh '.venv/bin/pip install build'
                sh '.venv/bin/python -m build'
            }
        }
        stage('Publish to PyPI') {
            when {
                buildingTag()
            }
            steps {
                withCredentials([string(credentialsId: 'pypi-token', variable: 'PYPI_TOKEN')]) {
                    sh '.venv/bin/pip install twine'
                    sh 'TWINE_USERNAME=__token__ TWINE_PASSWORD="$PYPI_TOKEN" .venv/bin/twine upload dist/*'
                }
            }
        }
    }
    post {
        always {
            sh 'docker compose logs > results/docker-logs.txt 2>&1 || true'
            sh 'docker compose down -v || true'
            junit 'results/*.xml'
            archiveArtifacts artifacts: 'results/**', allowEmptyArchive: true
            archiveArtifacts artifacts: 'dist/**', allowEmptyArchive: true
            cleanWs()
        }
    }
}Code language: Groovy (groovy)

Look at how far this file has come. In Part 3, the Jenkinsfile had three stages and no post block. Now it has seven stages, a post { always } block with five cleanup steps, pipeline-level options for timeout and concurrency, conditional stages for release builds, and secure credential injection for publishing. Every addition solved a specific problem you encountered along the way.

Commit and push the updated file:

cd ~/projects/helloci
git add Jenkinsfile
git commit -m "Add Publish to PyPI stage with secure credential injection"
git push origin mainCode language: Shell Session (shell)

---

8. Test with TestPyPI First

Before publishing to the real PyPI, test the entire flow against TestPyPI. TestPyPI is a separate instance of the package index that exists specifically for testing uploads. Packages uploaded there do not appear on the real PyPI.

Create a TestPyPI API token at test.pypi.org using the same process as Section 2. Store it in Jenkins as a separate credential with the ID testpypi-token.

Temporarily modify the Publish to PyPI stage to target TestPyPI:

stage('Publish to PyPI') {
    when {
        buildingTag()
    }
    steps {
        withCredentials([string(credentialsId: 'testpypi-token', variable: 'PYPI_TOKEN')]) {
            sh '.venv/bin/pip install twine'
            sh 'TWINE_USERNAME=__token__ TWINE_PASSWORD="$PYPI_TOKEN" .venv/bin/twine upload dist/* --repository testpypi'
        }
    }
}Code language: Groovy (groovy)

The --repository testpypi flag tells twine to upload to https://test.pypi.org/legacy/ instead of the production PyPI.

Tip: twine knows the TestPyPI URL by default — you do not need to configure it manually. The --repository testpypi flag is a built-in shortcut.

Tag a test release, push the tag, and verify the upload succeeds:

git tag v0.1.0-test
git push origin v0.1.0-testCode language: Shell Session (shell)

After the build completes, check test.pypi.org/project/helloci to confirm the package appears. Once verified, revert the stage back to use pypi-token and remove the --repository testpypi flag for production use.

---

9. Tag, Push, and Watch the Full Pipeline

With the production Jenkinsfile in place and the pypi-token credential stored, trigger the full pipeline by creating and pushing a version tag:

git tag v0.1.0
git push origin v0.1.0Code language: Shell Session (shell)

Open the Jenkins build console and watch all seven stages execute in sequence:

1. Setup Python — creates the virtual environment.
2. Install Dependencies — installs helloci and its test dependencies.
3. Lint — runs ruff against the source and test directories.
4. Unit Tests — runs pytest and generates JUnit XML.
5. Integration Tests — starts Docker Compose, waits for Postgres, and runs integration tests.
6. Build Release — builds the wheel and sdist into dist/.
7. Publish to PyPI — uploads dist/* to PyPI using the stored API token.

The post block then captures Docker logs, tears down Compose, archives results and dist artifacts, and cleans the workspace.

If any stage fails, the pipeline stops. The Publish to PyPI stage only runs if Build Release succeeds, which only runs if all tests and lint pass. This is the safety net — broken code never reaches PyPI.

---

10. Verify on PyPI

After the build completes successfully, confirm the package is live:

pip install helloci==0.1.0Code language: Shell Session (shell)

Collecting helloci==0.1.0
  Downloading helloci-0.1.0-py3-none-any.whl
Installing collected packages: helloci
Successfully installed helloci-0.1.0Code language: Shell Session (shell)

Run the installed package to verify it works:

helloci greet WorldCode language: Shell Session (shell)

Hello, World!Code language: Shell Session (shell)

You can also visit https://pypi.org/project/helloci/0.1.0/ in a browser to see the project page, which displays the package metadata, version history, and installation instructions.

Note: PyPI rejects uploads of a version that already exists. If you need to fix a problem in a released version, bump the version number in pyproject.toml, create a new tag, and push again. You cannot overwrite a published version.

---

11. Security Best Practices Recap

You have a working publish pipeline. Here is a checklist of security practices to maintain over time:

• Scope tokens to specific projects. Never use an account-wide token in CI. A project-scoped token limits the damage if it leaks.
• Use Secret Text, not plaintext. Jenkins Credentials encrypts the value at rest and masks it in logs. Never store tokens as build parameters, environment variables in the Jenkins UI, or hardcoded strings in the Jenkinsfile.
• Never echo credentials. Avoid echo, printenv, set -x, or any other command that could print the token to the console. Jenkins masks known secrets, but defense in depth means not relying on a single protection layer.
• Rotate tokens periodically. Revoke the old token on PyPI, create a new one, and update the Jenkins credential. Rotation limits the window of exposure if a token is silently compromised.
• Audit credential usage. Jenkins logs which builds accessed which credentials. Review these logs periodically to ensure only expected jobs are using the pypi-token credential.
• Restrict credential scope if needed. Jenkins supports folder-level and job-level credential scoping. If your Jenkins instance runs many projects, restrict pypi-token to the helloci job rather than making it globally available.

---

12. Series Recap — From Zero to PyPI in Eleven Parts

You started this series with nothing — no Jenkins, no pipeline, no automation. Eleven tutorials later, you have a production-grade CI/CD pipeline that takes every commit through a gauntlet of quality checks and publishes releases to PyPI with a single git tag.

Here is what you built, part by part:

• Part 1 — Prerequisites and Mental Model. You set up the development environment — Python, Docker, Jenkins — and learned how declarative pipelines map to the build lifecycle.
• Part 2 — Create the Minimal Repo. You built the helloci package from scratch with pyproject.toml, a greet() function, a CLI entry point, and a passing test suite.
• Part 3 — First Jenkinsfile. You wrote a three-stage Jenkinsfile, pointed Jenkins at the repository, and got your first green build.
• Part 4 — Test Reporting with JUnit. You added --junitxml to pytest and wired junit into the pipeline so Jenkins tracks test results over time.
• Part 5 — Fast-Fail with Lint. You added a Lint stage with ruff that runs before tests, catching style violations early and failing the build fast.
• Part 6 — Integration Tests with Docker Compose. You added a Postgres container, wrote integration tests that hit a real database, and managed container lifecycle inside the pipeline.
• Part 7 — Artifacts and Debugging. You captured Docker logs, archived test results and reports, and practiced a forensics workflow for diagnosing failed builds.
• Part 8 — Workspace Cleanup, Timeouts, and Retries. You hardened the pipeline with cleanWs(), a global timeout, and targeted retry logic for flaky infrastructure.
• Part 9 — Concurrency and Port Conflicts. You added disableConcurrentBuilds() and unique Compose project names to prevent parallel builds from colliding.
• Part 10 — Release Pipeline. You added a tag-triggered Build Release stage that produces wheels and sdists, turning the CI pipeline into a CI/CD pipeline.
• Part 11 — Publish to PyPI. You stored a PyPI API token securely in Jenkins, added a Publish stage with withCredentials, and shipped the package to the world.

The pipeline you have now is not a toy. It lints, tests at two levels, manages infrastructure, handles failures gracefully, builds release artifacts, and publishes them — all triggered by a git push. Every stage was added to solve a real problem, and every problem was explained before the solution.

The Jenkinsfile is about 70 lines long. It does everything.

---

Summary

You added the Publish to PyPI stage to the helloci pipeline, completing the CI/CD journey from source code to published package.

• A PyPI API token scoped to the helloci project provides the minimum permissions needed for publishing.
• Jenkins Credentials stores the token encrypted, and withCredentials injects it at runtime while masking it in build logs.
• The Publish to PyPI stage uses twine upload dist/* with token-based authentication via TWINE_USERNAME and TWINE_PASSWORD environment variables, and only runs on tagged builds alongside the Build Release stage.
• Testing against TestPyPI first with --repository testpypi catches upload issues before they affect the production index.
• The final Jenkinsfile contains seven stages, a comprehensive post block, and pipeline-level options for timeout and concurrency — everything built incrementally across eleven tutorials.

This is the end of the series. You have a complete, working CI/CD pipeline. Every commit gets linted and tested. Every tag gets built and published. Every secret stays secret. Ship it.