Installing and Running
Pre-Requisites
The first and most important requirement is that you have a running docker host that accepts HTTP(s) requests. If you have docker running locally, you probably are set.
In order to use fastlane, you also need to have both Mongo DB and Redis instances available.
IMPORTANT WARNING: If you are running fastlane on MacOS, you must expose docker host port to fastlane. This can be achieved by running the following command:
$ docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:1234:1234 bobrik/socat TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock
This will bridge the port 1234
in the container to the 1234
port in the host and allow us to use the default localhost:1234
docker host.
Installing
To install locally, you need python >=3.6.5(preferrably python >=3.7). Just run pip install fastlane
and you are good to go.
Required Services
fastlane is divided in two different parts: API
and Worker
.
The API
is responsible for receiving requests for new jobs, as well as providing task and job metadata.
The Worker
is responsible for starting and monitoring jobs in the docker farm.
Configuration
Both the API
and the Worker
can use the same configuration file. In order to get a sample configuration file that you can modify, run the following command:
$ fastlane config > my.conf
To configure Mongo DB, we can change the MONGODB_CONFIG
, or set it via environment variable.
MONGODB_CONFIG = {
"host": "mongodb://localhost:10101/fastlane",
"db": "fastlane",
"serverSelectionTimeoutMS": 100,
"connect": false
}
In order to connect to Redis, we must supply a valid connection url (either in configuration file or environment variable), likt this:
REDIS_URL = 'redis://localhost:10100/0'
Last but not least, you must configure your docker farm using the DOCKER_HOSTS
configuration variable (or environment variable):
DOCKER_HOSTS = [
{
"match": "",
"hosts": ["localhost:2375"],
"maxRunning":2
}
]
This configuration specifies that for all tasks with some task_id
that matches the match
configuration, these are the docker hosts that should be used, as well the maximum number of running containers that should be used. Another configuration might look like:
DOCKER_HOSTS = [
{
"match": "task-system1-.+",
"hosts": ["system1.docker.mycompany.com:2375"],
"maxRunning": 20
},
{
"match": "task-system2-.+",
"hosts": ["system2.docker.mycompany.com:2375"],
"maxRunning": 20
},
]
This configuration only allows tasks that start with task-system1-
or task-system2-
. Any other tasks won't be run. The empty match is a match-all case.
Running the API
$ fastlane api -vvv -c my.conf
In order to ensure that the API is working as expected, open in your browser http://localhost:10000/healthcheck
and http://localhost:10000/status
. The first ensures that the API has access to Mongo DB and redis. The second that the docker farm as well as the queues are working properly.
The API
log is structured as JSON. For more details type fastlane worker --help
.
Running the Workers
Just run the command:
$ fastlane worker -vvv -c my.conf
The worker command has options to define which queues it should monitor. This enabled users to scale each queue independently (jobs, monitor, etc.).
After this command, jobs should be running. The Worker
log is structured as JSON. For more details type fastlane worker --help
.