Setup Drone Server and Docker Runner for Gitea

Drone, a continiuous integration platform which is super close to my heart :heart: !

In this post we will setup Drone Server and Drone Runner for Gitea to run your Continuous Integration Pipelines, then we will setup a example pipeline, discover the drone-cli and how to extend our setup with more runners.

The source code of this post is available in my github repository

What is Drone?

Drone is a self-service continuous delivery platform which is built on container technology and can be used for CI/CD Pipelines, and has one extensive list of plugins enabling you to run almost any workflow you can think of.

With configuration as code, pipelines are configured with a simple, easy‑to‑read file that you commit to your git repository such as GitHub, Gitlab, Gogs, Gitea, Bitbucket and for this tutorial we will be focusing on Gitea.

Each Pipeline step is executed inside an isolated docker container that is automatically downloaded at runtime, if not found in cache.

Beautiful pipeline syntax:

---
kind: pipeline
type: docker
name: hello-world

steps:
  - name: say-hello
    image: busybox
    commands:
      - echo hello-world

What are we doing today

In a previous post we’ve setup gitea for version control and in this tutorial we will be setting up a drone server with ssl termination using traefik and letsencrypt, and a drone runner which will be responsible for running the jobs.

Then we will create a oauth application so that we can integrate drone with gitea, so that we can trigger our pipelines when we commit to gitea.

A note taken from readme.drone.io

Please note we strongly recommend installing Drone on a dedicated instance. We do not recommend installing Drone and Gitea on the same machine due to network complications, and we definitely do not recommend installing Drone and Gitea on the same machine using docker-compose.

For this example we will be running drone-server and drone-runner on the same instance for demonstration purposes on docker-compose, and Gitea running on a separate instance.

Assumptions

I will assume that you have docker and docker-compose installed. If you are following this guide to run Drone with Gitea, you can look at my tutorial on setting up Gitea on Docker with Traefik

If you need more info on Traefik you can have a look at their website, but I have also written a post on setting up Traefik v2 in detail, but we will touch on that in this post.

Drone Components

Before diving into the setup of Drone, we will give a small overview of components and terminology:

Drone Server: Responsible for hosting the UI, storing the encrypted secrets, hosts the API, etc.
Drone Runner: The component that will run your actual builds. There’s multiple runners that drone offers, but for this case we will focus on the docker runner.

Environment Details

I have 2 DNS entries pointing to 2 different instances:

Gitea: git.rbkr.xyz (gitea node: 192.168.0.10) - setup from this post
Drone: ci.rbkr.xyz (this node: 192.168.0.12) - this post
Drone Docker Runner: (this node: 192.168.0.12) - this post

Accessing our Drone Server will be done over HTTPS on port 443.

Create Oauth Application

We need to create a oauth application in Gitea in order for us to authenticate with our Gitea credentials in Drone.

Head over to your profile and select settings:

Select the “Applications” tab and you should see the following:

Create a new Oauth2 application for drone and set the redirect uri to your drone’s url

And select “Create application”:

You will get a client id and client secret:

Then save the results in your .env on your drone project directory, which we will create:

mkdir drone
cd drone

Since we are in the drone project directory, save the .env file with the following content (yours will differ):

DRONE_GITEA_CLIENT_ID=a46018de-3bd4-4e6c-86d6-825da0cb65c0
DRONE_GITEA_CLIENT_SECRET=tSjxsV9_wTzdYl33iLIStvyvNmnll5MsTA134zzGzKk=

Once you saved the content, we need to write the docker-compose.yml for drone-server and our drone-runner. You want to split your server and runners from each other, but since this is just a demonstration, we will keep them on the same docker host:

---
version: '3.8'

services:
  drone-traefik:
    image: traefik:2.4
    container_name: drone-traefik
    restart: unless-stopped
    volumes:
      - ./traefik/acme.json:/acme.json
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - public
    labels:
      - 'traefik.enable=true'
      - 'traefik.http.routers.api.rule=Host(`traefik.rbkr.xyz`)'
      - 'traefik.http.routers.api.entrypoints=https'
      - 'traefik.http.routers.api.service=api@internal'
      - 'traefik.http.routers.api.tls=true'
      - 'traefik.http.routers.api.tls.certresolver=letsencrypt'
    ports:
      - 80:80
      - 443:443
    command:
      - '--api'
      - '--providers.docker=true'
      - '--providers.docker.exposedByDefault=false'
      - '--entrypoints.http=true'
      - '--entrypoints.http.address=:80'
      - '--entrypoints.http.http.redirections.entrypoint.to=https'
      - '--entrypoints.http.http.redirections.entrypoint.scheme=https'
      - '--entrypoints.https=true'
      - '--entrypoints.https.address=:443'
      - '--certificatesResolvers.letsencrypt.acme.email=me@example.com'
      - '--certificatesResolvers.letsencrypt.acme.storage=acme.json'
      - '--certificatesResolvers.letsencrypt.acme.httpChallenge.entryPoint=http'
      - '--log=true'
      - '--log.level=INFO'
    logging:
      driver: "json-file"
      options:
        max-size: "1m"

  drone-server:
    container_name: drone
    image: drone/drone:${DRONE_VERSION:-2.4}
    restart: unless-stopped
    depends_on:
      drone-traefik:
        condition: service_started
    environment:
      # https://docs.drone.io/server/provider/gitea/
      - DRONE_DATABASE_DRIVER=sqlite3
      - DRONE_DATABASE_DATASOURCE=/data/database.sqlite
      - DRONE_GITEA_SERVER=https://git.rbkr.xyz/
      - DRONE_GIT_ALWAYS_AUTH=false
      - DRONE_RPC_SECRET=${DRONE_RPC_SECRET}
      - DRONE_SERVER_PROTO=https
      - DRONE_SERVER_HOST=ci.rbkr.xyz
      - DRONE_TLS_AUTOCERT=false
      - DRONE_USER_CREATE=${DRONE_USER_CREATE}
      - DRONE_GITEA_CLIENT_ID=${DRONE_GITEA_CLIENT_ID}
      - DRONE_GITEA_CLIENT_SECRET=${DRONE_GITEA_CLIENT_SECRET}
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.drone.rule=Host(`ci.rbkr.xyz`)"
      - "traefik.http.routers.drone.entrypoints=https"
      - "traefik.http.routers.drone.tls.certresolver=letsencrypt"
      - "traefik.http.routers.drone.service=drone-service"
      - "traefik.http.services.drone-service.loadbalancer.server.port=80"
    networks:
      - public
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./drone:/data

  drone-runner:
    container_name: drone-runner
    image: drone/drone-runner-docker:${DRONE_RUNNER_VERSION:-1}
    restart: unless-stopped
    depends_on:
      drone:
        condition: service_started
    environment:
      # https://docs.drone.io/runner/docker/installation/linux/
      # https://docs.drone.io/server/metrics/
      - DRONE_RPC_PROTO=https
      - DRONE_RPC_HOST=ci.rbkr.xyz
      - DRONE_RPC_SECRET=${DRONE_RPC_SECRET}
      - DRONE_RUNNER_NAME="${HOSTNAME}-runner"
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RUNNER_NETWORKS=public
      - DRONE_DEBUG=false
      - DRONE_TRACE=false
    networks:
      - public
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

networks:
  public:
    name: public

In the docker-compose.yml we have the following key items that we should highlight:

Traefik:

traefik.http.routers.api.rule=Host() - the host header we will use to access Traefik’s dashboard
80:80, 443:443 - exposing port 80 and 443 on the host
--certificatesResolvers.letsencrypt.acme.email - the email address for communication from LetsEncrypt

Drone Server:

DRONE_GITEA_SERVER - the git server url from my previous post
DRONE_RPC_SECRET - the shared secret used to authenticate the rpc connection to the server, which we saved to the .env file.
DRONE_SERVER_PROTO - set to https in my case as I am using Traefik for ssl termination
DRONE_SERVER_HOST - the hostname of my gitea server
DRONE_TLS_AUTOCERT - I have set this to false, as I am using Traefik, if you want to make use of LetsEncrypt set this to true
DRONE_USER_CREATE - we are using this to auto provision our system admin account.
DRONE_GITEA_CLIENT_ID - this is the gitea oauth client id that we created earlier and the value is saved in our .env file.
DRONE_GITEA_CLIENT_SECRET - his is the gitea oauth client secret that we created earlier and the value is saved in our .env file.
traefik.http.routers.drone.rule=Host() - the hostname we will route requests to the drone server, in my case ci.rbkr.xyz.

Drone Docker Runner

DRONE_RPC_PROTO - the protocol we use to communicate with the drone server, I used https.
DRONE_RPC_HOST - this is the drone server which we will communicate to, in my case ci.rbkr.xyz.
DRONE_RPC_SECRET - the shared secret used to authenticate the rpc connection to the server, which we saved to the .env file.
DRONE_RUNNER_NAME - the name of our runner, useful for identifying runners when you have multiple runners.
DRONE_RUNNER_CAPACITY - the number of concurrent pipelines that a runner can execute.
DRONE_RUNNER_NETWORKS - the list of docker networks thats attached to the pipeline steps.

And since we require a couple more environment variables, we can generate a DRONE_RPC_SECRET with openssl:

openssl rand -hex 24
faa1ce4fd08e481bb737548ac7569c202fa7001928056028

And then we can update our .env:

HOSTNAME=ip-172-31-16-4
GITEA_ADMIN_USER=ruanbekker
DRONE_RPC_SECRET=faa1ce4fd08e481bb737548ac7569c202fa7001928056028
DRONE_GITEA_CLIENT_ID=a46018de-3bd4-4e6c-86d6-825da0cb65c0
DRONE_GITEA_CLIENT_SECRET=tSjxsV9_wTzdYl33iLIStvyvNmnll5MsTA134zzGzKk=
DRONE_USER_CREATE="username:${GITEA_ADMIN_USER},machine:false,admin:true,token:${DRONE_RPC_SECRET}"

Once we have our configuration up to date and saved as docker-compose.yml, create the traefik directory and set the permissions of our acme.json:

mkdir traefik
touch traefik/acme.json
chmod 600 traefik/acme.json

Now its time to start our containers:

docker-compose up -d
...
Creating drone-traefik ... done
Creating drone-server ... done
Creating drone-runner ... done

Once they are started, allow Traefik about a minute or so to sort out the LetsEncrypt certificates, and then head over to your Drone Server UI, in my case it’s https://ci.rbkr.xyz and you should see a ui like this:

Select continue, as we are still logged into Gitea, it asks us to authorise this application:

Then continue with the registration:

Now you should see your repositories from git appearing in the dashboard (if you created any repositories that is)

Lets head back to our hello-world repository:

If you don’t have one you can create a repository, then commit this pipeline file in .drone.yml:

---
kind: pipeline
type: docker
name: hello-world

trigger:
  event:
    - push

steps:
  - name: say-hello
    image: busybox
    commands:
      - echo hello-world

So then we should have a file like this in Gitea:

Now head back to Drone where all the repositories are listed:

Then select the repository where you commited your .drone.yml, in my case its “ruanbekker/hello-world” and you should see that the repository is not active:

From the “Settings” tab, scroll down to the bottom and ensure that drone’s configuration is set to .drone.yml and select “Save changes”:

Head back to git, and commit a file, in my case it will be a file trigger just so that we can trigger the pipeline:

From Drone, when we look at our builds, we can see that our latest execution succeeded:

When we select our execution, from our log view we can see each step’s output:

We can also see our steps from the graph view:

For more examples, Drone has a repo called drone_playground, which we can import into Gitea:

Paste the Migrate / Clone URL https://github.com/drone/drone_playground and name your repository:

Then select “Migrate Repository”, and you should have a repository in your Gitea:

Head back to drone’s dashboard, select Sync to get the repository from Gitea, and you should see it in the list of repositories:

Select the repository, activate it and scroll to the bottom where you can provide the name of your .drone.yml and point it to mysql/.drone.yml as we want to use the mysql pipeline from Git, you can use any of the available pipelines, but I’m going with the mysql one:

So it will look like the following in the configuration section:

Then select “save changes”. Head back to Gitea and trigger the pipeline by pushing a commit to master or creating a new build via Drone, for this example I will be pushing a file trigger to master, then from Drone we should see that the pipeline was triggered:

If we select the execution, we can see that our pipeline ran successfully and from the steps we can see the following happened:

clone: fetches the content from git
mysql-server: starts up a mysql container
mysql healthcheck: ensures that the mysql container started and using a mysql client to connect to mysql
mysql-client DDL: creates a table in mysql and inserts data into mysql
mysql-client DML: reads the data from the table that we created

Drone CLI

We can interact with Drone using their CLI, for installation documentation you can consult their docs:

https://docs.drone.io/cli/install/

But in short for Mac, the installation steps look like this:

curl -L https://github.com/drone/drone-cli/releases/latest/download/drone_darwin_amd64.tar.gz | tar zx
sudo cp drone /usr/local/bin/drone

Verifying that the drone cli utility was installed:

drone --version
drone version 1.4.0

Now to configure drone cli, we need to get our personal access token by heading to our /account section, which for me is https://ci.rbkr.xyz/account

Which will look like the following:

When we first try out the example api usage command they provide:

$ curl -i https://ci.rbkr.xyz/api/user -H "Authorization: Bearer faa1ce4fd08e481bb737548ac7569c202fa7001928056028"
HTTP/2 200
cache-control: no-cache, no-store, must-revalidate, private, max-age=0
content-type: application/json
date: Thu, 28 Oct 2021 07:38:51 GMT
expires: Thu, 01 Jan 1970 00:00:00 UTC
pragma: no-cache
vary: Origin
x-accel-expires: 0
content-length: 263

{"id":1,"login":"ruanbekker","email":"ruan@localhost.net","machine":false,"admin":true,"active":true,"avatar":"https://git.rbkr.xyz/user/avatar/ruanbekker/-1","syncing":false,"synced":1635405616,"created":1635401206,"updated":1635401206,"last_login":1635401572}

And when we try out the example cli usage, set the drone server and drone token to your environment:

$ export DRONE_SERVER=https://ci.rbkr.xyz
$ export DRONE_TOKEN=faa1ce4fd08e481bb737548ac7569c202fa7001928056028

A list of subcommands are available when running:

$ drone --help
NAME:
   drone - command line utility

USAGE:
   drone [global options] command [command options] [arguments...]

VERSION:
   1.4.0

COMMANDS:
     build      manage builds
     cron       manage cron jobs
     log        manage logs
     encrypt    encrypt a secret
     exec       execute a local build
     info       show information about the current user
     repo       manage repositories
     user       manage users
     secret     manage secrets
     server     manage servers
     queue      queue operations
     orgsecret  manage organization secrets
     autoscale  manage autoscaling
     convert    convert legacy format
     lint       lint the yaml file
     sign       sign the yaml file
     jsonnet    generate .drone.yml from jsonnet
     starlark   generate .drone.yml from starlark
     plugins    plugin helper functions
     template   manage templates
     help, h    Shows a list of commands or help for one command

Then list our repositories:

$ drone repo ls
ruanbekker/drone_playground
ruanbekker/hello-world

List our builds for a repository:

$ drone build ls ruanbekker/hello-world
Build #4
Status: success
Event: push
Commit: a219a2602a62b2ffaac429eefcd95627ac2c792b
Branch: master
Ref: refs/heads/master
Author: ruanbekker <ruan@localhost.net>
Message: Update '.drone.yml'

We can view the logs of our build, let’s take the clone step as an example:

$ drone log view ruanbekker/hello-world 4 1 1
Initialized empty Git repository in /drone/src/.git/
+ git fetch origin +refs/heads/master:
From https://git.rbkr.xyz/ruanbekker/hello-world
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> origin/master
+ git checkout a219a2602a62b2ffaac429eefcd95627ac2c792b -b master
Already on 'master'

The nice thing that I really like about drone is to exec local builds using the cli. First clone your repo:

git clone https://git.rbkr.xyz/ruanbekker/hello-world

Change into the repo directory:

cd hello-world

Then execute a local build:

$ drone exec
[say-hello:0] + echo hello-world
[say-hello:1] hello-world

Add a secret to our repo:

drone secret add --name dummysecret --data dummyvalue ruanbekker/hello-world

List our secrets:

$ drone secret ls ruanbekker/hello-world
dummysecret
Pull Request Read:  false
Pull Request Write: false

Then update your pipeline step to:

...
steps:
  - name: say-hello
    image: busybox
    environment:
      DUMMYSECRET:
        from_secret: dummysecret
    commands:
      - if [ "$${DUMMYSECRET}" == "dummyvalue" ] ; then echo true; else echo false; fi

And when we commit it to git, we can see that we evaluated the value from the secret (as a basic example that the secret was published to drone):

Extending our Setup

If its a case where we want to run more nodes, with more runnes, we can implement the following docker-compose.yml for a extra runner:

---
version: '3.8'

services:
  drone-runner:
    container_name: drone-runner
    image: drone/drone-runner-docker:${DRONE_RUNNER_VERSION:-1}
    restart: unless-stopped
    environment:
      - DRONE_RPC_PROTO=https
      - DRONE_RPC_HOST=ci.rbkr.xyz
      - DRONE_RPC_SECRET=${DRONE_RPC_SECRET}
      - DRONE_RUNNER_NAME="${HOSTNAME}-runner"
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_RUNNER_NETWORKS=public
      - DRONE_DEBUG=false
      - DRONE_TRACE=false
    networks:
      - public
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

networks:
  public:
    name: public

Just remember to populate your environment variables with your own configuration.

We only demonstrated the docker runner, but have a look at their documentation as they have multiple runners available:

https://docs.drone.io/runner/overview/

It’s really easy extending your setup with drone plugins, this is also where drone shines, look at their extensive list of plugins:

http://plugins.drone.io/

Drone Announcements

There’s been a lot of changes recently at Drone, so I will try and keep this tutorial up to date, but things might change dependending when you read this.

For Drone announcements view:

https://blog.drone.io/tags/announcements/

For Drone documentation view:

https://readme.drone.io/

Thank You

I hope this was helpful, I was really impressed with Glitchtip. If you liked this content, please make sure to share or come say hi on my website or twitter:

Website: ruan.dev
Twitter: @ruanbekker

What is Drone?#

What are we doing today#

Assumptions#

Drone Components#

Environment Details#

Create Oauth Application#

Drone CLI#

Extending our Setup#

Drone Announcements#

Thank You#