Migrating from Skiff1

For reference Skiff1 docs can be found at Skiff1 docs.

This guide walks you through converting a standard Skiff1 repository to a Skiff2 compatible repository.

An application can be setup for Skiff2 while leaving it on Skiff1, allowing testing and validation that it works in Skiff2 before switching over.

See also Getting Started for creating a new Skiff2 application.

Register your project in skiff2​

Add a project-config.json file to the skiff2-projects/ directory in the Skiff-Commodore repo and open a pull request.

Name the file <your-project-name>.project-config.json:

{
    "name": "my-project",
    "ai2Project": "my-ai2-project",
    "primaryContact": "user:you@allenai.org",
    "projectAdmins": ["group:my-team@allenai.org"],
    "team": "my-team",
    "repo": "allenai/my-project",
    "allenaiDomains": false
}

Notes on the config:

• repo must already exist on GitHub before your PR is merged.
• The ReViz GitHub team must have at least Read access to repo (and any remoteRepos). We use this to confirm the repo exists when reviewing your PR and to help debug deploy issues. Add it under Settings → Collaborators and teams → Add teams in your repo.
• ai2Project valid values are listed in the AI2 project tag keys.
• projectAdmins and primaryContact will be made admins on the project.

See the project-config.json reference for all options.

Once your PR is merged, Skiff2 will automatically:

• Create a GCP project
• Provision SSL certificates and DNS records
• Set the following GitHub Actions variables on your repo:
  • SKIFF2_PROJECT_ID
  • SKIFF2_WORKLOAD_IDENTITY_PROVIDER
  • SKIFF2_SERVICE_ACCOUNT

Create skiff2.json​

Add a skiff2.json at the root of the repo to describe the services:

For a standard Skiff1 application based on the skiff-template, the skiff2.json would be:

{
    "$schema": "https://storage.googleapis.com/skiff2-general/skiff2-config.schema.json",
    "services": [
        {
            "name": "proxy",
            "cwd": "./proxy",
            "dockerFile": "prod.Dockerfile",
            "dependsOn": [
                "ui"
            ],
            "sidecars": [
                {
                    "name": "api",
                    "cwd": "./api"
                }
            ],
            "extraBuildArgs": [
                "--build-arg",
                "UI_IMAGE=gcr.io/${PROJECT_ID}/${REPO_NAME}-ui:${COMMIT_SHA}"
            ],
            "isRootService": true,
            "allowDelete": false,
            "secretFiles": {
                "SECRET_ENV_NAME": "/path/on/disk/filename.txt"
            }
        },
        {
            "name": "ui",
            "cwd": "./ui",
            "deploy": false
        }
    ]
}

• name — a short identifier for this service
• cwd — directory containing the Dockerfile, relative to skiff2.json
• isRootService: true — routes the top-level domain (my-project.apps.allenai.org) to this service

If you need custom DNS routing to the service (i.e. a TLD) you can set customDomains.

See the skiff2.json reference for all available options.

{
    "services": [
        {
            "name": "proxy",
            "machine": {
                "minInstances": 1,
                "cpuIdle": false
            }
        }
    ]
}

Check each Dockerfile has a CMD​

In Skiff1, the start command was often supplied by Kubernetes via the command field in .skiff/webapp.jsonnet rather than baked into the image, so some repos have Dockerfiles with no CMD or ENTRYPOINT. Skiff2 runs your image directly on Cloud Run — without a CMD, the container will fail to start.

For each service, open its Dockerfile and confirm there is a CMD (or ENTRYPOINT). If there isn't, copy the command from the matching container in .skiff/webapp.jsonnet:

{
    name: fullyQualifiedName + '-api',
    image: apiImage,
    command: [ '/api/start.sh' ],
    ...
}

becomes:

CMD ["/api/start.sh"]

Carry over --build-arg flags​

If your repo has a clouddeploy-build.yml (or similar Cloud Build config), check the docker build step for any --build-arg flags. Skiff1 passed these at build time; Skiff2 needs them in extraBuildArgs on the relevant service in skiff2.json.

For example, a Cloud Build step like:

args:
    - 'build'
    - '--build-arg'
    - 'BUILDKIT_INLINE_CACHE=1'
    - '--build-arg'
    - 'UI_IMAGE=gcr.io/$PROJECT_ID/$REPO_NAME-ui:$COMMIT_SHA'
    - '--build-arg'
    - 'CONF_FILE=prod.conf'

becomes:

{
    "services": [
        {
            "name": "proxy",
            "extraBuildArgs": [
                "--build-arg", "UI_IMAGE=gcr.io/${PROJECT_ID}/${REPO_NAME}-ui:${COMMIT_SHA}",
                "--build-arg", "CONF_FILE=prod.conf"
            ]
        }
    ]
}

You can drop BUILDKIT_INLINE_CACHE=1 — Skiff2 handles build caching for you.

Setup Secrets​

There are two kinds of secrets to consider when migrating. For a general overview of how secrets work in Skiff2, see the Secrets guide.

Runtime Secrets​

Runtime secrets are needed while the app is running — for example, API keys or database credentials that were previously stored in Marina.

To migrate them:

1. Open your application in Marina and navigate to the secrets section.
2. Open the webapp.jsonnet file in your repo.
3. Open Google Secret Manager for your new Skiff2 project.

Your webapp.jsonnet references Marina secrets in a format like this:

{
    name: fullyQualifiedName + '-api',
    image: apiImage,
    command: [ '/api/start.sh' ],
    ...
    env: [
        {
            name: "API_KEY", // Use this as the ENV_VARIABLE_NAME in Secret Manager
            valueFrom: {
                secretKeyRef: {
                    name: "some-secret-secrets", // Marina-only identifier — not used in Secret Manager
                    key: "API_KEY" // Marina-only identifier — not used in Secret Manager
                }
            }
        }
    ]
},

Use the name and key fields to locate the corresponding secrets in Marina. Copy each secret's value into Secret Manager, naming it with the following format:

<ENV_NAME>-<SERVICE_NAME>-<ENV_VARIABLE_NAME>

For example:

prod-proxy-API_KEY

Mounting Secrets as Files​

If your app also mounts secrets as files, you'll need to create them in Secret Manager (same as above) and additionally configure secretFiles in your skiff2.json.

Look for a volumeMounts section in your webapp.jsonnet:

{
    name: fullyQualifiedName + '-api',
    image: apiImage,
    command: [ '/api/start.sh' ],
    ...
    volumeMounts: [
                    {
                        name: 'cfg', // Find this in Marina
                        mountPath: '/secret/cfg', // Prefix for path used in skiff2.json
                        readOnly: true
                    },
    ]
}

For each volume mount, find the corresponding secret value in Marina and create it in Secret Manager using the same naming format.

Then mount the secret as a file by adding a secretFiles field to your service in skiff2.json:

{
    "services": [
        {
            "name": "proxy",
            "secretFiles": {
                "API_KEY": "/secret/cfg/filename.txt"
            }
        }
    ]
}

• The key is the ENV_VARIABLE_NAME of the secret you created in Secret Manager.
• The value is the file path where the secret will be written inside your container.

Build-time Secrets​

Build-time secrets are needed during docker build — for example, credentials for a private package registry or SSH keys. Store them as GitHub Actions secrets on your repo and pass them to the build action:

- name: Skiff2 Build
  uses: allenai/skiff2-actions/shared-actions/build@main
  with:
      project_id: ${{ vars.SKIFF2_PROJECT_ID }}
      secrets: |
          my-secret=${{ secrets.MY_SECRET }}

To access the secret in your Dockerfile, use RUN --mount=type=secret,id=my-secret ....

See the build action reference for all secret input formats.

Authentication​

Skiff1 used oauth2-proxy for authentication (login: "ai2" in skiff.json). Skiff2 uses Identity-Aware Proxy (IAP), which is provisioned automatically for every service.

By default, only users with an @allenai.org account can access your app — no configuration needed for internal tools.

For public-facing apps (previously using no login), set allowUnauthenticated: true on the service in skiff2.json:

{
    "services": [
        {
            "name": "proxy",
            "allowUnauthenticated": true
        }
    ]
}

If your app reads the authenticated user's email from a request header, the header name has changed:

• Skiff1 (oauth2-proxy): X-Auth-Request-Email — value is the bare email
• Skiff2 (IAP): X-Goog-Authenticated-User-Email — value is prefixed with an identity namespace (e.g. accounts.google.com:user@allenai.org) that must be stripped

Read more on Authentication guide.

Set up GitHub Actions​

Create a workflow file at .github/workflows/deploy.yml:

name: Skiff2 Build & Deploy
on:
    push:
        branches:
            - main # will need to add more branches here if you plan on deploying to other environments.

# needed to authenticate with google
permissions:
    contents: read
    id-token: write

concurrency:
    group: ${{ github.workflow }}-${{ github.ref }}
    cancel-in-progress: false

jobs:
    build_deploy:
        runs-on: ubuntu-latest
        environment:
            name: ${{ github.ref_name }}
            url: ${{ steps.deploy-services.outputs.default_url }}
        steps:
            - name: Skiff2 Setup
              id: setup
              uses: allenai/skiff2-actions/shared-actions/setup@main
              with:
                  workload_identity_provider: ${{ vars.SKIFF2_WORKLOAD_IDENTITY_PROVIDER }} # prepopulated on project setup
                  service_account: ${{ vars.SKIFF2_SERVICE_ACCOUNT }} # prepopulated on project setup
                  project_id: ${{ vars.SKIFF2_PROJECT_ID }} # prepopulated on project setup

            - name: Skiff2 Build
              id: build
              uses: allenai/skiff2-actions/shared-actions/build@main
              with:
                  project_id: ${{ vars.SKIFF2_PROJECT_ID }}

            - name: Skiff2 Deploy Infra
              if: github.ref_name == 'main'
              uses: allenai/skiff2-actions/shared-actions/deploy-infra@main
              with:
                  project_id: ${{ vars.SKIFF2_PROJECT_ID }}
                  command: apply

            - name: Skiff2 Deploy Services
              id: deploy-services
              uses: allenai/skiff2-actions/shared-actions/deploy-services@main
              with:
                  project_id: ${{ vars.SKIFF2_PROJECT_ID }}
                  command: apply

The four steps do the following:

1. Setup — Authenticates to GCP using Workload Identity Federation and configures Docker for Container Registry
2. Build — Builds your Docker images and pushes them to GCR
3. Deploy Infra — Runs Terraform to update shared infrastructure (load balancer, SSL certificates, DNS). Only runs on main.
4. Deploy Services — Runs Terraform to update your Cloud Run services

Push to main and test​

Once merged, your workflow will run and your app will be live at:

Validate your app is working correctly on pandajungle.org before proceeding.

Production deploy​

Once you're happy with the app on Skiff2, complete the migration:

Enable your production domains by removing allenaiDomains: false from your project-config.json (or setting it to true) and opening a PR. Once merged, allen.ai and apps.allenai.org will point to Skiff2 instead of Skiff1.

Migrate any custom domains from skiff.json's customDomains field. For each custom domain:

• Add it to customDomains on the relevant service in skiff2.json (SSL cert + routing)
• If it's a managed zone domain (allen.ai or apps.allenai.org), also add it to customDomains in your project-config.json (DNS record)

See the skiff2.json customDomains reference for details.