Make patch version

fix AttributeError for ruamel.yaml

.github/issue_template.md

@@ -2,7 +2,7 @@
 
 Before raising an issue here, answer the following questions for yourself, please:
 
-* Did you read through the troubleshooting section? (https://github.com/ninech/netbox-docker/#troubleshooting)
+* Did you read through the troubleshooting section? (https://github.com/netbox-community/netbox-docker/#troubleshooting)
 * Have you updated to the latest version and tried again? (i.e. `git pull` and `docker-compose pull`)
 * Have you reset the project and tried again? (i.e. `docker-compose down -v`)
 * Are you confident that your problem is related to the Docker or Docker Compose setup this project provides?

DOCKER_HUB.md

@@ -0,0 +1,66 @@
+# cloud.docker.com Configuration
+
+The automatic build is configured in cloud.docker.com.
+
+The following build configuration is expected:
+
+```yaml
+Source Repository: github.com/netbox-community/netbox-docker
+Build Location: Build on Docker Hub's infrastructure
+Autotest: Internal and External Pull Requests
+Repository Links: Enable for Base Image
+Build Rules:
+- Source Type: Branch
+  Source: master
+  Docker Tag: branches-main
+  Dockerfile location: Dockerfile
+- Source Type: Branch
+  Source: master
+  Docker Tag: branches-ldap
+  Dockerfile location: Dockerfile.ldap
+- Source Type: Branch
+  Source: master
+  Docker Tag: prerelease-main
+  Dockerfile location: Dockerfile
+- Source Type: Branch
+  Source: master
+  Docker Tag: prerelease-ldap
+  Dockerfile location: Dockerfile.ldap
+- Source Type: Branch
+  Source: master
+  Docker Tag: release-main
+  Dockerfile location: Dockerfile
+- Source Type: Branch
+  Source: master
+  Docker Tag: release-ldap
+  Dockerfile location: Dockerfile.ldap
+Build Environment Variables:
+# Create an app on Github and use it's OATH credentials here
+- Key: GITHUB_OAUTH_CLIENT_ID
+  Value: <secret>
+- Key: GITHUB_OAUTH_CLIENT_SECRET
+  Value: <secret>
+Build Triggers:
+- Name: Cron Trigger
+# Use this trigger in combination with e.g. https://cron-job.org in order to regularly schedule builds
+```
+
+## Background Knowledge
+
+The build system of cloud.docker.com is not made for this kind of project.
+But we found a way to make it work, and this is how:
+
+1. The docker hub build system [allows to overwrite the scripts that get executed
+   for `build`, `test` and `push`](overwrite). See `hooks/*`.
+2. Shared functionality of the scripts `build`, `test` and `push` is extracted to `hooks/common`.
+3. The `build` script runs `run_build()` from `hooks/common`.
+   This triggers either `build-branches.sh`, `build-latest.sh` or directly `build.sh`.
+4. The `test` script just invokes `docker-compose` commands.
+5. The `push` script runs `run_build()` from `hooks/common` with a `--push-only` flag.
+   This causes the `build.sh` script to not re-build the Docker image, but just the just built image.
+
+The _Docker Tag_ configuration setting is misused to select the type (_release_, _prerelease_, _branches_) of the build as well as the variant (_main_, _ldap_).
+
+The _Dockerfile location_ configuration setting is completely ignored by the build scripts.
+
+[overwrite]: https://docs.docker.com/docker-hub/builds/advanced/#override-build-test-or-push-commands

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.6-alpine3.8
+FROM python:3.7-alpine3.10
 
 RUN apk add --no-cache \
       bash \
@@ -21,28 +21,42 @@ RUN pip install \
 # napalm is used for gathering information from network devices
       napalm \
 # ruamel is used in startup_scripts
-      ruamel.yaml \
+      'ruamel.yaml>=0.15,<0.16' \
 # pinning django to the version required by netbox
 # adding it here, to install the correct version of
 # django-rq
-      'Django>=1.11,<2.1' \
+      'Django>=2.2,<2.3' \
 # django-rq is used for webhooks
       django-rq
 
+ARG BRANCH=master
+
+WORKDIR /tmp
+
+# As the requirements don't change very often,
+# and as they take some time to compile,
+# we try to cache them very agressively.
+ARG REQUIREMENTS_URL=https://raw.githubusercontent.com/netbox-community/netbox/$BRANCH/requirements.txt
+ADD ${REQUIREMENTS_URL} requirements.txt
+RUN pip install -r requirements.txt
+
+# Cache bust when the upstream branch changes:
+# ADD will fetch the file and check if it has changed
+# If not, Docker will use the existing build cache.
+# If yes, Docker will bust the cache and run every build step from here on.
+ARG REF_URL=https://api.github.com/repos/netbox-community/netbox/contents?ref=$BRANCH
+ADD ${REF_URL} version.json
+
 WORKDIR /opt
 
-ARG BRANCH=master
-ARG URL=https://github.com/digitalocean/netbox/archive/$BRANCH.tar.gz
+ARG URL=https://github.com/netbox-community/netbox/archive/$BRANCH.tar.gz
 RUN wget -q -O - "${URL}" | tar xz \
   && mv netbox* netbox
 
-WORKDIR /opt/netbox
-RUN pip install -r requirements.txt
-
 COPY docker/configuration.docker.py /opt/netbox/netbox/netbox/configuration.py
 COPY configuration/gunicorn_config.py /etc/netbox/config/
 COPY docker/nginx.conf /etc/netbox-nginx/nginx.conf
-COPY docker/docker-entrypoint.sh docker-entrypoint.sh
+COPY docker/docker-entrypoint.sh /opt/netbox/docker-entrypoint.sh
 COPY startup_scripts/ /opt/netbox/startup_scripts/
 COPY initializers/ /opt/netbox/initializers/
 COPY configuration/configuration.py /etc/netbox/config/configuration.py

Dockerfile.ldap

@@ -1,4 +1,4 @@
-ARG DOCKER_ORG=ninech
+ARG DOCKER_ORG=netboxcommunity
 ARG DOCKER_REPO=netbox
 ARG FROM_TAG=latest
 FROM $DOCKER_ORG/$DOCKER_REPO:$FROM_TAG

README.md

@@ -1,31 +1,29 @@
 # netbox-docker
 
-[![Build Status](https://travis-ci.org/ninech/netbox-docker.svg?branch=master)][travis]
+[The Github repository](netbox-docker-github) houses the components needed to build Netbox as a Docker container.
+Images are built using this code are released to [Docker Hub][netbox-dockerhub] every night.
 
-This repository houses the components needed to build NetBox as a Docker container.
-Images built using this code are released to [Docker Hub][netbox-dockerhub] every night.
+Questions? Before opening an issue on Github, please join the [Network To Code][ntc-slack] Slack and ask for help in our `#netbox-docker` channel.
 
-Questions? Before opening an issue on Github, please join the [Network To Code][ntc-slack] and ask for help in our `#netbox-docker` channel.
-
-[travis]: https://travis-ci.org/ninech/netbox-docker
-[netbox-dockerhub]: https://hub.docker.com/r/ninech/netbox/tags/
+[netbox-dockerhub]: https://hub.docker.com/r/netboxcommunity/netbox/tags/
+[netbox-docker-github]:  https://github.com/netbox-community/netbox-docker/
 [ntc-slack]: http://slack.networktocode.com/
 
 ## Quickstart
 
-To get NetBox up and running:
+To get Netbox up and running:
 
-```
-$ git clone -b master https://github.com/ninech/netbox-docker.git
-$ cd netbox-docker
-$ docker-compose pull
-$ docker-compose up -d
+```bash
+git clone -b master https://github.com/netbox-community/netbox-docker.git
+cd netbox-docker
+docker-compose pull
+docker-compose up -d
 ```
 
 The application will be available after a few minutes.
 Use `docker-compose port nginx 8080` to find out where to connect to.
 
-```
+```bash
 $ echo "http://$(docker-compose port nginx 8080)/"
 http://0.0.0.0:32768/
 
@@ -44,7 +42,7 @@ Default credentials:
 * Password: **admin**
 * API Token: **0123456789abcdef0123456789abcdef01234567**
 
-[docker-reception]: https://github.com/ninech/reception
+[docker-reception]: https://github.com/nxt-engineering/reception
 
 ## Dependencies
 
@@ -62,10 +60,10 @@ These are defined in `netbox.env`.
 Read [Environment Variables in Compose][compose-env] to understand about the various possibilities to overwrite these variables.
 (The easiest solution being simply adjusting that file.)
 
-To find all possible variables, have a look at the [configuration.docker.py][docker-config] and [docker-entrypoint.sh][entrypoint] files.
-Generally, the environment variables are called the same as their respective NetBox configuration variables.
+To find all possible variables, have a look at the [configuration.py][docker-config] and [docker-entrypoint.sh][entrypoint] files.
+Generally, the environment variables are called the same as their respective Netbox configuration variables.
 Variables which are arrays are usually composed by putting all the values into the same environment variables with the values separated by a whitespace ("` `").
-For example defining `ALLOWED_HOSTS=localhost ::1 127.0.0.1` would allows access to NetBox through `http://localhost:8080`, `http://[::1]:8080` and `http://127.0.0.1:8080`.
+For example defining `ALLOWED_HOSTS=localhost ::1 127.0.0.1` would allows access to Netbox through `http://localhost:8080`, `http://[::1]:8080` and `http://127.0.0.1:8080`.
 
 [compose-env]: https://docs.docker.com/compose/environment-variables/
 
@@ -74,7 +72,7 @@ For example defining `ALLOWED_HOSTS=localhost ::1 127.0.0.1` would allows access
 The default settings are optimized for (local) development environments.
 You should therefore adjust the configuration for production setups, at least the following variables:
 
-* `ALLOWED_HOSTS`: Add all URLs that lead to your NetBox instance, space separated. E.g. `ALLOWED_HOSTS=netbox.mycorp.com server042.mycorp.com 2a02:123::42 10.0.0.42 localhost ::1 127.0.0.1` (It's good advice to always allow localhost connections for easy debugging, i.e. `localhost ::1 127.0.0.1`.)
+* `ALLOWED_HOSTS`: Add all URLs that lead to your Netbox instance, space separated. E.g. `ALLOWED_HOSTS=netbox.mycorp.com server042.mycorp.com 2a02:123::42 10.0.0.42 localhost ::1 127.0.0.1` (It's good advice to always allow localhost connections for easy debugging, i.e. `localhost ::1 127.0.0.1`.)
 * `DB_*`: Use your own persistent database. Don't use the default passwords!
 * `EMAIL_*`: Use your own mailserver.
 * `MAX_PAGE_SIZE`: Use the recommended default of 1000.
@@ -85,7 +83,7 @@ You should therefore adjust the configuration for production setups, at least th
 
 You may run this image in a cluster such as Docker Swarm, Kubernetes or OpenShift, but this is advanced level.
 
-In this case, we encourage you to statically configure NetBox by starting from [NetBox's example config file][default-config], and mounting it into your container in the directory `/etc/netbox/config/` using the mechanism provided by your container platform (i.e. [Docker Swarm configs][swarm-config], [Kubernetes ConfigMap][k8s-config], [OpenShift ConfigMaps][openshift-config]).
+In this case, we encourage you to statically configure Netbox by starting from [Netbox's example config file][default-config], and mounting it into your container in the directory `/etc/netbox/config/` using the mechanism provided by your container platform (i.e. [Docker Swarm configs][swarm-config], [Kubernetes ConfigMap][k8s-config], [OpenShift ConfigMaps][openshift-config]).
 
 But if you rather continue to configure your application through environment variables, you may continue to use [the built-in configuration file][docker-config].
 We discourage storing secrets in environment variables, as environment variable are passed on to all sub-processes and may leak easily into other systems, e.g. error collecting tools that often collect all environment variables whenever an error occurs.
@@ -101,12 +99,13 @@ If a secret is defined by an environment variable and in the respective file at
 * `EMAIL_PASSWORD`: `/run/secrets/email_password`
 * `NAPALM_PASSWORD`: `/run/secrets/napalm_password`
 * `REDIS_PASSWORD`: `/run/secrets/redis_password`
+* `AUTH_LDAP_BIND_PASSWORD`: `/run/secrets/auth_ldap_bind_password`
 
-Please also consider [the advice about running NetBox in production](#production) above!
+Please also consider [the advice about running Netbox in production](#production) above!
 
-[docker-config]: https://github.com/ninech/netbox-docker/blob/master/docker/configuration.docker.py
-[default-config]: https://github.com/digitalocean/netbox/blob/develop/netbox/netbox/configuration.example.py
-[entrypoint]: https://github.com/ninech/netbox-docker/blob/master/docker/docker-entrypoint.sh
+[docker-config]: https://github.com/netbox-community/netbox-docker/blob/master/configuration/configuration.py
+[default-config]: https://github.com/netbox-community/netbox/blob/develop/netbox/netbox/configuration.example.py
+[entrypoint]: https://github.com/netbox-community/netbox-docker/blob/master/docker/docker-entrypoint.sh
 [swarm-config]: https://docs.docker.com/engine/swarm/configs/
 [swarm-secrets]: https://docs.docker.com/engine/swarm/secrets/
 [openshift-config]: https://docs.openshift.org/latest/dev_guide/configmaps.html
@@ -116,9 +115,9 @@ Please also consider [the advice about running NetBox in production](#production
 
 ### NAPALM Configuration
 
-Since v2.1.0 NAPALM has been tightly integrated into NetBox.
-NAPALM allows NetBox to fetch live data from devices and return it to a requester via its REST API.
-To learn more about what NAPALM is and how it works, please see the documentation from the [libary itself][napalm-doc] or the documentation from [NetBox][netbox-napalm-doc] on how it is integrated.
+Since v2.1.0 NAPALM has been tightly integrated into Netbox.
+NAPALM allows Netbox to fetch live data from devices and return it to a requester via its REST API.
+To learn more about what NAPALM is and how it works, please see the documentation from the [libary itself][napalm-doc] or the documentation from [Netbox][netbox-napalm-doc] on how it is integrated.
 
 To enable this functionality, simply complete the following lines in `netbox.env` (or appropriate secrets mechanism) :
 
@@ -133,21 +132,22 @@ However, if you don't need this functionality, leave these blank.
 
 ### Customizable Reporting
 
-NetBox includes [customized reporting][netbox-reports-doc] that allows the user to write Python code and determine the validity of the data within NetBox.
+Netbox includes [customized reporting][netbox-reports-doc] that allows the user to write Python code and determine the validity of the data within Netbox.
 The `REPORTS_ROOT` variable is setup as a mapped directory within this Docker container to `/reports/` and includes the example directly from the documentation for `devices.py`.
-However, it has been renamed to `devices.py.example` which prevents NetBox from recognizing it as a valid report.
+However, it has been renamed to `devices.py.example` which prevents Netbox from recognizing it as a valid report.
 This was done to avoid unnessary issues from being opened when the default does not work for someone's expectations.
 
 To re-enable this default report, simply rename `devices.py.example` to `devices.py` and browse within the WebUI to `/extras/reports/`.
-You can also dynamically add any other report to this same directory and NetBox will be able to see it without restarting the container.
+You can also dynamically add any other report to this same directory and Netbox will be able to see it without restarting the container.
 
 [netbox-reports-doc]: https://netbox.readthedocs.io/en/stable/additional-features/reports/
 
 ### Custom Initialization Code (e.g. Automatically Setting Up Custom Fields)
 
 When using `docker-compose`, all the python scripts present in `/opt/netbox/startup_scripts` will automatically be executed after the application boots in the context of `./manage.py`.
+The execution of the startup scripts can be prevented by setting the environment variable `SKIP_STARTUP_SCRIPTS` to `true`, e.g. in the file `env/netbox.env`.
 
-That mechanism can be used for many things, e.g. to create NetBox custom fields:
+That mechanism can be used for many things, e.g. to create Netbox custom fields:
 
 ```python
 # docker/startup_scripts/load_custom_fields.py
@@ -173,7 +173,7 @@ if created:
 
 #### Initializers
 
-Initializers are built-in startup scripts for defining NetBox custom fields, groups, users and many other resources.
+Initializers are built-in startup scripts for defining Netbox custom fields, groups, users and many other resources.
 All you need to do is to mount you own `initializers` folder ([see `docker-compose.yml`][netbox-docker-compose]).
 Look at the [`initializers` folder][netbox-docker-initializers] to learn how the files must look like.
 
@@ -197,8 +197,8 @@ text_field:
   - virtualization.models.VirtualMachine
 ```
 
-[netbox-docker-initializers]: https://github.com/ninech/netbox-docker/tree/master/initializers
-[netbox-docker-compose]: https://github.com/ninech/netbox-docker/blob/master/docker-compose.yml
+[netbox-docker-initializers]: https://github.com/netbox-community/netbox-docker/tree/master/initializers
+[netbox-docker-compose]: https://github.com/netbox-community/netbox-docker/blob/master/docker-compose.yml
 
 ##### Available Groups for User/Group initializers
 
@@ -211,12 +211,12 @@ echo "from django.contrib.auth.models import Permission\nfor p in Permission.obj
 
 #### Custom Docker Image
 
-You can also build your own NetBox Docker image containing your own startup scripts, custom fields, users and groups
+You can also build your own Netbox Docker image containing your own startup scripts, custom fields, users and groups
 like this:
 
-```
+```Dockerfile
 ARG VERSION=latest
-FROM ninech/netbox:$VERSION
+FROM netboxcommunity/netbox:$VERSION
 
 COPY startup_scripts/ /opt/netbox/startup_scripts/
 COPY initializers/ /opt/netbox/initializers/
@@ -224,32 +224,32 @@ COPY initializers/ /opt/netbox/initializers/
 
 ## Netbox Version
 
-The `docker-compose.yml` file is prepared to run a specific version of NetBox.
+The `docker-compose.yml` file is prepared to run a specific version of Netbox.
 To use this feature, set the environment-variable `VERSION` before launching `docker-compose`, as shown below.
 `VERSION` may be set to the name of
-[any tag of the `ninech/netbox` Docker image on Docker Hub][netbox-dockerhub].
+[any tag of the `netboxcommunity/netbox` Docker image on Docker Hub][netbox-dockerhub].
 
-```
-$ export VERSION=v2.2.6
-$ docker-compose pull netbox
-$ docker-compose up -d
+```bash
+export VERSION=v2.2.6
+docker-compose pull netbox
+docker-compose up -d
 ```
 
-You can also build a specific version of the NetBox image. This time, `VERSION` indicates any valid
-[Git Reference][git-ref] declared on [the 'digitalocean/netbox' Github repository][netbox-github].
+You can also build a specific version of the Netbox image. This time, `VERSION` indicates any valid
+[Git Reference][git-ref] declared on [the 'netbox-community/netbox' Github repository][netbox-github].
 Most commonly you will specify a tag or branch name.
 
-```
-$ export VERSION=develop
-$ docker-compose build --no-cache netbox
-$ docker-compose up -d
+```bash
+export VERSION=develop
+docker-compose build --no-cache netbox
+docker-compose up -d
 ```
 
 Hint: If you're building a specific version by tag name, the `--no-cache` argument is not strictly necessary.
 This can increase the build speed if you're just adjusting the config, for example.
 
 [git-ref]: https://git-scm.com/book/en/v2/Git-Internals-Git-References
-[netbox-github]: https://github.com/digitalocean/netbox/releases
+[netbox-github]: https://github.com/netbox-community/netbox/releases
 
 ### LDAP enabled variant
 
@@ -262,7 +262,7 @@ Custom values can be injected using environment variables, similar to the main c
 This section is a collection of some common issues and how to resolve them.
 If your issue is not here, look through [the existing issues][issues] and eventually create a new issue.
 
-[issues]: (https://github.com/ninech/netbox-docker/issues)
+[issues]: (https://github.com/netbox-community/netbox-docker/issues)
 
 ### Docker Compose basics
 
@@ -271,7 +271,7 @@ If your issue is not here, look through [the existing issues][issues] and eventu
   Running `docker-compose logs -f netbox` will just show the logs for netbox.
 * You can stop everything using `docker-compose stop`.
 * You can clean up everything using `docker-compose down -v --remove-orphans`. **This will also remove any related data.**
-* You can enter the shell of the running NetBox container using `docker-compose exec netbox /bin/bash`. Now you have access to `./manage.py`, e.g. to reset a password.
+* You can enter the shell of the running Netbox container using `docker-compose exec netbox /bin/bash`. Now you have access to `./manage.py`, e.g. to reset a password.
 * To access the database run `docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'`
 * To create a database backup run `docker-compose exec postgres sh -c 'pg_dump -cU $POSTGRES_USER $POSTGRES_DB' | gzip > db_dump.sql.gz`
 * To restore that database backup run `gunzip -c db_dump.sql.gz | docker exec -i $(docker-compose ps -q postgres) sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'`.
@@ -299,12 +299,12 @@ Now start everything up again.
 If this didn't help, try to see if there's anything in the logs indicating why nginx doesn't start:
 
 ```bash
-$ docker-compose logs -f nginx
+docker-compose logs -f nginx
 ```
 
 ### Getting a "Bad Request (400)"
 
-> When connecting to the NetBox instance, I get a "Bad Request (400)" error.
+> When connecting to the Netbox instance, I get a "Bad Request (400)" error.
 
 This usually happens when the `ALLOWED_HOSTS` variable is not set correctly.
 
@@ -326,14 +326,14 @@ docker-compose up -d netbox netbox-worker
 First make sure that the webhooks feature is enabled in your Netbox configuration and that a redis host is defined.
 Check `netbox.env` if the following variables are defined:
 
-```
+```bash
 WEBHOOKS_ENABLED=true
 REDIS_HOST=redis
 ```
 
 Then make sure that the `redis` container and at least one `netbox-worker` are running.
 
-```
+```bash
 # check the container status
 $ docker-compose ps
 
@@ -370,64 +370,77 @@ docker-compose run --rm -T redis sh -c 'redis-cli -h redis -a $REDIS_PASSWORD mo
 
 If you don't see anything happening after you triggered a webhook, double-check the configuration of the `netbox` and the `netbox-worker` containers and also check the configuration of your webhook in the admin interface of Netbox.
 
-### Breaking Changes
+## Breaking Changes
 
 From time to time it might become necessary to re-engineer the structure of this setup.
 Things like the `docker-compose.yml` file or your Kubernetes or OpenShift configurations have to be adjusted as a consequence.
 Since April 2018 each image built from this repo contains a `NETBOX_DOCKER_PROJECT_VERSION` label.
-You can check the label of your local image by running `docker inspect ninech/netbox:v2.3.1 --format "{{json .ContainerConfig.Labels}}"`.
+You can check the label of your local image by running `docker inspect netboxcommunity/netbox:v2.3.1 --format "{{json .ContainerConfig.Labels}}"`.
 Compare the version with the list below to check whether a breaking change was introduced with that version.
 
 The following is a list of breaking changes of the `netbox-docker` project:
 
+* 0.17.0: Updated the python image to `python:3.7-alpine3.10` in [#144][144]. Fixed the permissions and group scripts for Netbox 2.6. in [#148][148].
+* 0.16.0: Update the Netbox URL from "github.com/digitalocean/netbox" to "github.com/netbox-community/netbox"
+* 0.15.0: Update for Netbox v2.6.0.
+  The `configuration/configuration.py` file has been updated to match the file from Netbox.
+  `CORS_ORIGIN_WHITELIST` has a new default value of `http://localhost`.
+  To provide a nice development environment, `CORS_ORIGIN_ALLOW_ALL` added to `env/netbox.env` with a default value of `True`.
+  There are also new options:
+  * `REDIS_CACHE_DATABASE`
+  * `CACHE_TIMEOUT` (set to 0 to disable caching)
+  * `CHANGELOG_RETENTION`
+  * `CORS_ORIGIN_REGEX_WHITELIST` (space separated list of regular expressions)
+  * `EXEMPT_VIEW_PERMISSIONS` (space separated list)
+  * `METRICS_ENABLED`
+* 0.14.0: Improved caching strategy [#137][137] [#136][136].
+  New `AUTH_LDAP_GROUP_TYPE` environment variable [#135][135].
+* 0.13.0: `AUTH_LDAP_BIND_PASSWORD` can now be extracted into a secrets file. [#133][133]
+* 0.12.0: A new flag `REDIS_SSL=false` was added to the `env/netbox.env` file. [#129][129]
+* 0.11.0: The docker-compose file now marks volumes as shared (`:z`). This should prevent SELinux problems [#131][131]
+* 0.9.0: Upgrade to at least 2.1.5
+* 0.8.0: Alpine linux was upgraded to 3.9 [#126][126]
 * 0.7.0: The value of the `MAX_PAGE_SIZE` environment variable was changed to `1000`, which is the default of Netbox.
 * 0.6.0: The naming of the default startup_scripts were changed.
   If you overwrite them, you may need to adjust these scripts.
 * 0.5.0: Alpine was updated to 3.8, `*.env` moved to `/env` folder
 * 0.4.0: In order to use Netbox webhooks you need to add Redis and a netbox-worker to your docker-compose.yml.
 * 0.3.0: Field `filterable: <boolean` was replaced with field `filter_logic: loose/exact/disabled`. It will default to `CF_FILTER_LOOSE=loose` when not defined.
-* 0.2.0: Re-organized paths: `/etc/netbox -> /etc/netbox/config` and `/etc/reports -> /etc/netbox/reports`. Fixes [#54](https://github.com/ninech/netbox-docker/issues/54).
+* 0.2.0: Re-organized paths: `/etc/netbox -> /etc/netbox/config` and `/etc/reports -> /etc/netbox/reports`. Fixes [#54][54].
 * 0.1.0: Introduction of the `NETBOX_DOCKER_PROJECT_VERSION`. (Not a breaking change per se.)
 
+[54]: https://github.com/netbox-community/netbox-docker/issues/54
+[126]: https://github.com/netbox-community/netbox-docker/pull/126
+[131]: https://github.com/netbox-community/netbox-docker/pull/131
+[129]: https://github.com/netbox-community/netbox-docker/pull/129
+[133]: https://github.com/netbox-community/netbox-docker/pull/133
+[135]: https://github.com/netbox-community/netbox-docker/pull/135
+[136]: https://github.com/netbox-community/netbox-docker/pull/136
+[137]: https://github.com/netbox-community/netbox-docker/pull/137
+[144]: https://github.com/netbox-community/netbox-docker/pull/144
+[148]: https://github.com/netbox-community/netbox-docker/pull/148
+
 ## Rebuilding & Publishing images
 
-`./build.sh` is used to rebuild the Docker image:
-
-```
-$ ./build.sh --help
-Usage: ./build.sh <branch> [--push]
-  branch  The branch or tag to build. Required.
-  --push  Pushes built Docker image to docker hub.
-
-You can use the following ENV variables to customize the build:
-  BRANCH   The branch to build.
-           Also used for tagging the image.
-  DOCKER_REPO The Docker registry (i.e. hub.docker.com/r/DOCKER_REPO/netbox)
-           Also used for tagging the image.
-           Default: ninech
-  SRC_REPO Which fork of netbox to use (i.e. github.com/<SRC_REPO>/netbox).
-           Default: digitalocean
-  URL      Where to fetch the package from.
-           Must be a tar.gz file of the source code.
-           Default: https://github.com/${SRC_REPO}/netbox/archive/$BRANCH.tar.gz
-```
+`./build.sh` can be used to rebuild the Docker image. See `./build.sh --help` for more information.
 
 ### Publishing Docker Images
 
-New Docker Images are built and published every 24h by using travis:
+New Docker images are built and published every 24h on the [Docker Build Infrastructure][docker-build-infra].
+`DOCKER_HUB.md` contains more information about the build infrastructure.
 
-[![Build Status](https://travis-ci.org/ninech/netbox-docker.svg?branch=master)][travis]
+[docker-build-infra]: https://hub.docker.com/r/netboxcommunity/netbox/builds/
 
 ## Tests
 
-To run the tests coming with NetBox, use the `docker-compose.yml` file as such:
+To run the tests coming with Netbox, use the `docker-compose.yml` file as such:
 
-```
-$ docker-compose run netbox ./manage.py test
+```bash
+docker-compose run netbox ./manage.py test
 ```
 
 ## About
 
-This repository is currently maintained and funded by [nine](https://nine.ch), your cloud navigator.
+This repository is currently maintained and funded by [nxt][nxt].
 
-[![logo of the company 'nine'](https://logo.apps.at-nine.ch/Dmqied_eSaoBMQwk3vVgn4UIgDo=/trim/500x0/logo_claim.png)](https://www.nine.ch)
+[nxt]: https://nxt.engineering/en/

VERSION

@@ -1 +1 @@
-0.7.0
+0.17.1

build-all.sh

@@ -12,9 +12,9 @@ BUILDS=("${BUILD:-"${ALL_BUILDS[@]}"}")
 
 echo "⚙️ Configured builds: ${BUILDS[*]}"
 
-VARIANTS=("" "ldap")
+VARIANTS=("main" "ldap")
 
-if [ ! -z "${DEBUG}" ]; then
+if [ -n "${DEBUG}" ]; then
   export DEBUG
 fi
 
@@ -22,67 +22,63 @@ ERROR=0
 
 # Don't build if not on `master` and don't build if on a pull request,
 # but build when DEBUG is not empty
-if [ ! -z "${DEBUG}" ] || \
-  ( [ "$TRAVIS_BRANCH" = "master" ] && [ "$TRAVIS_PULL_REQUEST" = "false" ] ); then
-  for VARIANT in "${VARIANTS[@]}"; do
-		export VARIANT
+for VARIANT in "${VARIANTS[@]}"; do
+  export VARIANT
 
-    # Checking which VARIANT to build
-    if [ -z "$VARIANT" ]; then
-      DOCKERFILE="Dockerfile"
-    else
-      DOCKERFILE="Dockerfile.${VARIANT}"
+  # Checking which VARIANT to build
+  if [ "${VARIANT}" == "main" ]; then
+    DOCKERFILE="${DOCKERFILE_PATH-Dockerfile}"
+  else
+    DOCKERFILE="${DOCKERFILE_PATH-Dockerfile}.${VARIANT}"
 
-      # Fail fast
-      if [ ! -f "${DOCKERFILE}" ]; then
-        echo "🚨 The Dockerfile '${DOCKERFILE}' for variant '${VARIANT}' doesn't exist."
-        ERROR=1
+    # Fail fast
+    if [ ! -f "${DOCKERFILE}" ]; then
+      echo "🚨 The Dockerfile '${DOCKERFILE}' for variant '${VARIANT}' doesn't exist."
+      ERROR=1
 
-        if [ -z "$DEBUG" ]; then
-          continue
-        else
-          echo "⚠️ Would skip this, but DEBUG is enabled."
-        fi
+      if [ -z "$DEBUG" ]; then
+        continue
+      else
+        echo "⚠️ Would skip this, but DEBUG is enabled."
       fi
     fi
+  fi
 
-    for BUILD in "${BUILDS[@]}"; do
-      echo "🛠 Building '$BUILD' from '$DOCKERFILE'"
-      case $BUILD in
-        release)
-          # build the latest release
-          # shellcheck disable=SC2068
-          ./build-latest.sh $@ || ERROR=1
-          ;;
-        prerelease)
-          # build the latest pre-release
-          # shellcheck disable=SC2068
-      		PRERELEASE=true ./build-latest.sh $@ || ERROR=1
-          ;;
-        branches)
-          # build all branches
-          # shellcheck disable=SC2068
-      		./build-branches.sh $@ || ERROR=1
-          ;;
-        special)
-          # special build
-          # shellcheck disable=SC2068
-      		#SRC_ORG=lampwins TAG=webhooks-backend ./build.sh "feature/webhooks-backend" $@ || ERROR=1
-          ;;
-        *)
-          echo "🚨 Unrecognized build '$BUILD'."
+  for BUILD in "${BUILDS[@]}"; do
+    echo "🛠 Building '$BUILD' from '$DOCKERFILE'"
+    case $BUILD in
+      release)
+        # build the latest release
+        # shellcheck disable=SC2068
+        ./build-latest.sh $@ || ERROR=1
+        ;;
+      prerelease)
+        # build the latest pre-release
+        # shellcheck disable=SC2068
+        PRERELEASE=true ./build-latest.sh $@ || ERROR=1
+        ;;
+      branches)
+        # build all branches
+        # shellcheck disable=SC2068
+        ./build-branches.sh $@ || ERROR=1
+        ;;
+      special)
+        # special build
+        # shellcheck disable=SC2068
+        #SRC_ORG=lampwins TAG=webhooks-backend ./build.sh "feature/webhooks-backend" $@ || ERROR=1
+        echo "✅ No special builds today."
+        ;;
+      *)
+        echo "🚨 Unrecognized build '$BUILD'."
 
-          if [ -z "$DEBUG" ]; then
-            exit 1
-          else
-            echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
-          fi
-          ;;
-      esac
-    done
+        if [ -z "$DEBUG" ]; then
+          exit 1
+        else
+          echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
+        fi
+        ;;
+    esac
   done
-else
-  echo "❎ Not building anything."
-fi
+done
 
 exit $ERROR

build-branches.sh

@@ -3,7 +3,7 @@
 
 echo "▶️ $0 $*"
 
-if [ ! -z "${GITHUB_OAUTH_CLIENT_ID}" ] && [ ! -z "${GITHUB_OAUTH_CLIENT_SECRET}" ]; then
+if [ -n "${GITHUB_OAUTH_CLIENT_ID}" ] && [ -n "${GITHUB_OAUTH_CLIENT_SECRET}" ]; then
   echo "🗝 Performing authenticated Github API calls."
   GITHUB_OAUTH_PARAMS="client_id=${GITHUB_OAUTH_CLIENT_ID}&client_secret=${GITHUB_OAUTH_CLIENT_SECRET}"
 else
@@ -11,7 +11,7 @@ else
   GITHUB_OAUTH_PARAMS=""
 fi
 
-ORIGINAL_GITHUB_REPO="digitalocean/netbox"
+ORIGINAL_GITHUB_REPO="${SRC_ORG-netbox-community}/${SRC_REPO-netbox}"
 GITHUB_REPO="${GITHUB_REPO-$ORIGINAL_GITHUB_REPO}"
 URL_RELEASES="https://api.github.com/repos/${GITHUB_REPO}/branches?${GITHUB_OAUTH_PARAMS}"
 

build-latest.sh

@@ -3,7 +3,7 @@
 
 echo "▶️ $0 $*"
 
-if [ ! -z "${GITHUB_OAUTH_CLIENT_ID}" ] && [ ! -z "${GITHUB_OAUTH_CLIENT_SECRET}" ]; then
+if [ -n "${GITHUB_OAUTH_CLIENT_ID}" ] && [ -n "${GITHUB_OAUTH_CLIENT_SECRET}" ]; then
   echo "🗝 Performing authenticated Github API calls."
   GITHUB_OAUTH_PARAMS="client_id=${GITHUB_OAUTH_CLIENT_ID}&client_secret=${GITHUB_OAUTH_CLIENT_SECRET}"
 else
@@ -11,7 +11,7 @@ else
   GITHUB_OAUTH_PARAMS=""
 fi
 
-ORIGINAL_GITHUB_REPO="digitalocean/netbox"
+ORIGINAL_GITHUB_REPO="netbox-community/netbox"
 GITHUB_REPO="${GITHUB_REPO-$ORIGINAL_GITHUB_REPO}"
 URL_RELEASES="https://api.github.com/repos/${GITHUB_REPO}/releases?${GITHUB_OAUTH_PARAMS}"
 
@@ -47,7 +47,7 @@ if [ "${PRERELEASE}" == "true" ]; then
 fi
 
 # Check if that version is not already available on docker hub:
-ORIGINAL_DOCKERHUB_REPO="ninech/netbox"
+ORIGINAL_DOCKERHUB_REPO="${DOCKER_ORG-netboxcommunity}/${DOCKER_REPO-netbox}"
 DOCKERHUB_REPO="${DOCKERHUB_REPO-$ORIGINAL_DOCKERHUB_REPO}"
 URL_DOCKERHUB_TOKEN="https://auth.docker.io/token?service=registry.docker.io&scope=repository:${DOCKERHUB_REPO}:pull"
 BEARER_TOKEN="$($CURL "${URL_DOCKERHUB_TOKEN}" | jq -r .token)"
@@ -55,7 +55,7 @@ BEARER_TOKEN="$($CURL "${URL_DOCKERHUB_TOKEN}" | jq -r .token)"
 URL_DOCKERHUB_TAG="https://registry.hub.docker.com/v2/${DOCKERHUB_REPO}/tags/list"
 AUTHORIZATION_HEADER="Authorization: Bearer ${BEARER_TOKEN}"
 
-if [ -z "$VARIANT" ]; then
+if [ -z "$VARIANT" ] || [ "$VARIANT" == "main" ]; then
   DOCKER_TAG="${VERSION}"
 else
   DOCKER_TAG="${VERSION}-${VARIANT}"
@@ -63,7 +63,11 @@ fi
 
 ALREADY_BUILT="$($CURL -H "${AUTHORIZATION_HEADER}" "${URL_DOCKERHUB_TAG}" | jq -e ".tags | any(.==\"${DOCKER_TAG}\")")"
 
-if [ "$ALREADY_BUILT" == "false" ]; then
+if [ -n "$DEBUG" ] || [ "$ALREADY_BUILT" == "false" ]; then
+  if [ -n "$DEBUG" ]; then
+    echo "⚠️ Would not build, because ${DOCKER_TAG} already exists on https://hub.docker.com/r/${DOCKERHUB_REPO}, but DEBUG is enabled."
+  fi
+  
   # shellcheck disable=SC2068
   ./build.sh "${VERSION}" $@
   exit $?

build.sh

@@ -6,9 +6,10 @@ echo "▶️ $0 $*"
 set -e
 
 if [ "${1}x" == "x" ] || [ "${1}" == "--help" ] || [ "${1}" == "-h" ]; then
-  echo "Usage: ${0} <branch> [--push]"
-  echo "  branch  The branch or tag to build. Required."
-  echo "  --push  Pushes built Docker image to docker hub."
+  echo "Usage: ${0} <branch> [--push|--push-only]"
+  echo "  branch       The branch or tag to build. Required."
+  echo "  --push       Pushes built the Docker image to the registry."
+  echo "  --push-only  Does not build. Only pushes the Docker image to the registry."
   echo ""
   echo "You can use the following ENV variables to customize the build:"
   echo "  DEBUG    If defined, the script does not stop when certain checks are unsatisfied."
@@ -26,15 +27,18 @@ if [ "${1}x" == "x" ] || [ "${1}" == "--help" ] || [ "${1}" == "-h" ]; then
   echo "             Else:          same as <BRANCH>"
   echo "  DOCKER_ORG The Docker registry (i.e. hub.docker.com/r/<DOCKER_ORG>/<DOCKER_REPO>) "
   echo "           Also used for tagging the image."
-  echo "           Default: ninech"
+  echo "           Default: netboxcommunity"
   echo "  DOCKER_REPO The Docker registry (i.e. hub.docker.com/r/<DOCKER_ORG>/<DOCKER_REPO>) "
   echo "           Also used for tagging the image."
   echo "           Default: netbox"
   echo "  DOCKER_TAG The name of the tag which is applied to the image."
   echo "           Useful for pushing into another registry than hub.docker.com."
   echo "           Default: <DOCKER_ORG>/<DOCKER_REPO>:<BRANCH>"
+  echo "  DOCKER_SHORT_TAG The name of the short tag which is applied to the image."
+  echo "           This is used to tag all patch releases to their containing version e.g. v2.5.1 -> v2.5"
+  echo "           Default: <DOCKER_ORG>/<DOCKER_REPO>:\$MAJOR.\$MINOR"
   echo "  SRC_ORG  Which fork of netbox to use (i.e. github.com/<SRC_ORG>/<SRC_REPO>)."
-  echo "           Default: digitalocean"
+  echo "           Default: netbox-community"
   echo "  SRC_REPO The name of the netbox for to use (i.e. github.com/<SRC_ORG>/<SRC_REPO>)."
   echo "           Default: netbox"
   echo "  URL      Where to fetch the package from."
@@ -44,13 +48,14 @@ if [ "${1}x" == "x" ] || [ "${1}" == "--help" ] || [ "${1}" == "-h" ]; then
   echo "           The value will be used as a suffix to the \$TAG and for the Dockerfile"
   echo "           selection. The TAG being build must exist for the base variant and"
   echo "           corresponding Dockerfile must start with the following lines:"
-  echo "             ARG DOCKER_ORG=ninech"
+  echo "             ARG DOCKER_ORG=netboxcommunity"
   echo "             ARG DOCKER_REPO=netbox"
   echo "             ARG FROM_TAG=latest"
   echo "             FROM \$DOCKER_ORG/\$DOCKER_REPO:\$FROM_TAG"
   echo "           Example: VARIANT=ldap will result in the tag 'latest-ldap' and the"
-  echo "            Dockerfile 'Dockerfile.ldap' being used."
-  echo "           Default: empty"
+  echo "           Dockerfile './Dockerfile.ldap' being used."
+  echo "           Exception: VARIANT=main will use the './Dockerfile' Dockerfile"
+  echo "           Default: main"
   echo "  HTTP_PROXY The proxy to use for http requests."
   echo "           Example: http://proxy.domain.tld:3128"
   echo "           Default: empty"
@@ -76,13 +81,32 @@ fi
 NETBOX_DOCKER_PROJECT_VERSION="${NETBOX_DOCKER_PROJECT_VERSION-$(sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' VERSION)}"
 
 # variables for fetching the source
-SRC_ORG="${SRC_ORG-digitalocean}"
+SRC_ORG="${SRC_ORG-netbox-community}"
 SRC_REPO="${SRC_REPO-netbox}"
 BRANCH="${1}"
 URL="${URL-https://github.com/${SRC_ORG}/${SRC_REPO}/archive/$BRANCH.tar.gz}"
 
+# Checking which VARIANT to build
+VARIANT="${VARIANT-main}"
+if [ "$VARIANT" == "main" ]; then
+  DOCKERFILE="Dockerfile"
+else
+  DOCKERFILE="Dockerfile.${VARIANT}"
+fi
+
+# Fail fast
+if [ ! -f "${DOCKERFILE}" ]; then
+  echo "🚨 The Dockerfile ${DOCKERFILE} for variant '${VARIANT}' doesn't exist."
+
+  if [ -z "$DEBUG" ]; then
+    exit 1
+  else
+    echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
+  fi
+fi
+
 # variables for tagging the docker image
-DOCKER_ORG="${DOCKER_ORG-ninech}"
+DOCKER_ORG="${DOCKER_ORG-netboxcommunity}"
 DOCKER_REPO="${DOCKER_REPO-netbox}"
 case "${BRANCH}" in
   master)
@@ -92,24 +116,20 @@ case "${BRANCH}" in
   *)
     TAG="${TAG-$BRANCH}";;
 esac
+
 DOCKER_TAG="${DOCKER_TAG-${DOCKER_ORG}/${DOCKER_REPO}:${TAG}}"
-
-# Checking which VARIANT to build
-if [ -z "$VARIANT" ]; then
-  DOCKERFILE="Dockerfile"
-else
-  DOCKERFILE="Dockerfile.${VARIANT}"
+if [ "$VARIANT" != "main" ]; then
   DOCKER_TAG="${DOCKER_TAG}-${VARIANT}"
+fi
 
-  # Fail fast
-  if [ ! -f "${DOCKERFILE}" ]; then
-    echo "🚨 The Dockerfile ${DOCKERFILE} for variant '${VARIANT}' doesn't exist."
+if [[ "${TAG}" =~ ^v([0-9]+)\.([0-9]+)\.[0-9]+$ ]]; then
+  MAJOR=${BASH_REMATCH[1]}
+  MINOR=${BASH_REMATCH[2]}
 
-    if [ -z "$DEBUG" ]; then
-      exit 1
-    else
-      echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
-    fi
+  DOCKER_SHORT_TAG="${DOCKER_SHORT_TAG-${DOCKER_ORG}/${DOCKER_REPO}:v${MAJOR}.${MINOR}}"
+
+  if [ "$VARIANT" != "main" ]; then
+    DOCKER_SHORT_TAG="${DOCKER_SHORT_TAG}-${VARIANT}"
   fi
 fi
 
@@ -117,8 +137,8 @@ DOCKER_OPTS=("${DOCKER_OPTS[@]}")
 
 # caching is only ok for version tags
 case "${TAG}" in
-  v*) ;;
-  *)  DOCKER_OPTS+=( "--no-cache" ) ;;
+ v*) ;;
+ *)  DOCKER_OPTS+=( "--no-cache" ) ;;
 esac
 
 DOCKER_OPTS+=( "--pull" )
@@ -152,12 +172,26 @@ else
   DOCKER_CMD="echo docker"
 fi
 
-echo "🐳 Building the Docker image '${DOCKER_TAG}' from the url '${URL}'."
-$DOCKER_CMD build -t "${DOCKER_TAG}" "${DOCKER_BUILD_ARGS[@]}" "${DOCKER_OPTS[@]}" -f "${DOCKERFILE}" .
-echo "✅ Finished building the Docker images '${DOCKER_TAG}'"
+if [ "${2}" != "--push-only" ] ; then
+  echo "🐳 Building the Docker image '${DOCKER_TAG}' from the url '${URL}'."
+  $DOCKER_CMD build -t "${DOCKER_TAG}" "${DOCKER_BUILD_ARGS[@]}" "${DOCKER_OPTS[@]}" -f "${DOCKERFILE}" .
+  echo "✅ Finished building the Docker images '${DOCKER_TAG}'"
 
-if [ "${2}" == "--push" ] ; then
+  if [ -n "$DOCKER_SHORT_TAG" ]; then
+    echo "🐳 Tagging image '${DOCKER_SHORT_TAG}'."
+    $DOCKER_CMD tag "${DOCKER_TAG}" "${DOCKER_SHORT_TAG}"
+    echo "✅ Tagged image '${DOCKER_SHORT_TAG}'"
+  fi
+fi
+
+if [ "${2}" == "--push" ] || [ "${2}" == "--push-only" ] ; then
   echo "⏫ Pushing '${DOCKER_TAG}"
   $DOCKER_CMD push "${DOCKER_TAG}"
   echo "✅ Finished pushing the Docker image '${DOCKER_TAG}'."
+
+  if [ -n "$DOCKER_SHORT_TAG" ]; then
+    echo "⏫ Pushing '${DOCKER_SHORT_TAG}'"
+    $DOCKER_CMD push "${DOCKER_SHORT_TAG}"
+    echo "✅ Finished pushing the Docker image '${DOCKER_SHORT_TAG}'."
+  fi
 fi

configuration/configuration.py

@@ -1,8 +1,9 @@
 import os
+import re
 import socket
 
 # For reference see http://netbox.readthedocs.io/en/latest/configuration/mandatory-settings/
-# Based on https://github.com/digitalocean/netbox/blob/develop/netbox/netbox/configuration.example.py
+# Based on https://github.com/netbox-community/netbox/blob/develop/netbox/netbox/configuration.example.py
 
 # Read secret from file
 def read_secret(secret_name):
@@ -44,6 +45,17 @@ DATABASE = {
 # https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-SECRET_KEY
 SECRET_KEY = os.environ.get('SECRET_KEY', read_secret('secret_key'))
 
+# Redis database settings. The Redis database is used for caching and background processing such as webhooks
+REDIS = {
+    'HOST': os.environ.get('REDIS_HOST', 'localhost'),
+    'PORT': int(os.environ.get('REDIS_PORT', 6379)),
+    'PASSWORD': os.environ.get('REDIS_PASSWORD', read_secret('redis_password')),
+    'DATABASE': os.environ.get('REDIS_DATABASE', '0'),
+    'CACHE_DATABASE': os.environ.get('REDIS_CACHE_DATABASE', '1'),
+    'DEFAULT_TIMEOUT': os.environ.get('REDIS_TIMEOUT', '300'),
+    'SSL': os.environ.get('REDIS_SSL', 'False').lower() == 'true',
+}
+
 #########################
 #                       #
 #   Optional settings   #
@@ -68,14 +80,18 @@ BANNER_LOGIN = os.environ.get('BANNER_LOGIN', '')
 # BASE_PATH = 'netbox/'
 BASE_PATH = os.environ.get('BASE_PATH', '')
 
+# Cache timeout in seconds. Set to 0 to dissable caching. Defaults to 900 (15 minutes)
+CACHE_TIMEOUT = int(os.environ.get('CACHE_TIMEOUT', 900))
+
+# Maximum number of days to retain logged changes. Set to 0 to retain changes indefinitely. (Default: 90)
+CHANGELOG_RETENTION = int(os.environ.get('CHANGELOG_RETENTION', 90))
+
 # API Cross-Origin Resource Sharing (CORS) settings. If CORS_ORIGIN_ALLOW_ALL is set to True, all origins will be
 # allowed. Otherwise, define a list of allowed origins using either CORS_ORIGIN_WHITELIST or
 # CORS_ORIGIN_REGEX_WHITELIST. For more information, see https://github.com/ottoyiu/django-cors-headers
 CORS_ORIGIN_ALLOW_ALL = os.environ.get('CORS_ORIGIN_ALLOW_ALL', 'False').lower() == 'true'
-CORS_ORIGIN_WHITELIST = os.environ.get('CORS_ORIGIN_WHITELIST', '').split(' ')
-CORS_ORIGIN_REGEX_WHITELIST = [
-    # r'^(https?://)?(\w+\.)?example\.com$',
-]
+CORS_ORIGIN_WHITELIST = list(filter(None, os.environ.get('CORS_ORIGIN_WHITELIST', 'https://localhost').split(' ')))
+CORS_ORIGIN_REGEX_WHITELIST = [re.compile(r) for r in list(filter(None, os.environ.get('CORS_ORIGIN_REGEX_WHITELIST', '').split(' ')))]
 
 # Set to True to enable server debugging. WARNING: Debugging introduces a substantial performance penalty and may reveal
 # sensitive information about your installation. Only enable debugging while performing testing. Never enable debugging
@@ -97,6 +113,10 @@ EMAIL = {
 # set ENFORCE_GLOBAL_UNIQUE to True.
 ENFORCE_GLOBAL_UNIQUE = os.environ.get('ENFORCE_GLOBAL_UNIQUE', 'False').lower() == 'true'
 
+# Exempt certain models from the enforcement of view permissions. Models listed here will be viewable by all users and
+# by anonymous users. List models in the form `<app>.<model>`. Add '*' to this list to exempt all models.
+EXEMPT_VIEW_PERMISSIONS = list(filter(None, os.environ.get('EXEMPT_VIEW_PERMISSIONS', '').split(' ')))
+
 # Enable custom logging. Please see the Django documentation for detailed guidance on configuring custom logs:
 #   https://docs.djangoproject.com/en/1.11/topics/logging/
 LOGGING = {}
@@ -105,10 +125,6 @@ LOGGING = {}
 # are permitted to access most data in NetBox (excluding secrets) but not make any changes.
 LOGIN_REQUIRED = os.environ.get('LOGIN_REQUIRED', 'False').lower() == 'true'
 
-# Base URL path if accessing NetBox within a directory. For example, if installed at http://example.com/netbox/, set:
-# BASE_PATH = 'netbox/'
-BASE_PATH = os.environ.get('BASE_PATH', '')
-
 # Setting this to True will display a "maintenance mode" banner at the top of every page.
 MAINTENANCE_MODE = os.environ.get('MAINTENANCE_MODE', 'False').lower() == 'true'
 
@@ -121,6 +137,9 @@ MAX_PAGE_SIZE = int(os.environ.get('MAX_PAGE_SIZE', 1000))
 # the default value of this setting is derived from the installed location.
 MEDIA_ROOT = os.environ.get('MEDIA_ROOT', os.path.join(BASE_DIR, 'media'))
 
+# Expose Prometheus monitoring metrics at the HTTP endpoint '/metrics'
+METRICS_ENABLED = os.environ.get('METRICS_ENABLED', 'False').lower() == 'true'
+
 # Credentials that NetBox will use to access live devices.
 NAPALM_USERNAME = os.environ.get('NAPALM_USERNAME', '')
 NAPALM_PASSWORD = os.environ.get('NAPALM_PASSWORD', read_secret('napalm_password'))
@@ -138,20 +157,6 @@ PAGINATE_COUNT = int(os.environ.get('PAGINATE_COUNT', 50))
 # When determining the primary IP address for a device, IPv6 is preferred over IPv4 by default. Set this to True to
 # prefer IPv4 instead.
 PREFER_IPV4 = os.environ.get('PREFER_IPV4', 'False').lower() == 'true'
-
-# The Webhook event backend is disabled by default. Set this to True to enable it. Note that this requires a Redis
-# database be configured and accessible by NetBox (see `REDIS` below).
-WEBHOOKS_ENABLED = os.environ.get('WEBHOOKS_ENABLED', 'False').lower() == 'true'
-
-# Redis database settings (optional). A Redis database is required only if the webhooks backend is enabled.
-REDIS = {
-    'HOST': os.environ.get('REDIS_HOST', 'localhost'),
-    'PORT': os.environ.get('REDIS_PORT', 6379),
-    'PASSWORD': os.environ.get('REDIS_PASSWORD', read_secret('redis_password')),
-    'DATABASE': os.environ.get('REDIS_DATABASE', '0'),
-    'DEFAULT_TIMEOUT': os.environ.get('REDIS_TIMEOUT', '300'),
-}
-
 # The file path where custom reports will be stored. A trailing slash is not needed. Note that the default value of
 # this setting is derived from the installed location.
 REPORTS_ROOT = os.environ.get('REPORTS_ROOT', '/etc/netbox/reports')
@@ -159,6 +164,10 @@ REPORTS_ROOT = os.environ.get('REPORTS_ROOT', '/etc/netbox/reports')
 # Time zone (default: UTC)
 TIME_ZONE = os.environ.get('TIME_ZONE', 'UTC')
 
+# The Webhook event backend is disabled by default. Set this to True to enable it. Note that this requires a Redis
+# database be configured and accessible by NetBox (see `REDIS` below).
+WEBHOOKS_ENABLED = os.environ.get('WEBHOOKS_ENABLED', 'False').lower() == 'true'
+
 # Date/time formatting. See the following link for supported formats:
 # https://docs.djangoproject.com/en/dev/ref/templates/builtins/#date
 DATE_FORMAT = os.environ.get('DATE_FORMAT', 'N j, Y')

configuration/ldap_config.py

@@ -1,7 +1,26 @@
 import ldap
 import os
 
-from django_auth_ldap.config import LDAPSearch, GroupOfNamesType
+from django_auth_ldap.config import LDAPSearch
+from importlib import import_module
+
+# Read secret from file
+def read_secret(secret_name):
+    try:
+        f = open('/run/secrets/' + secret_name, 'r', encoding='utf-8')
+    except EnvironmentError:
+        return ''
+    else:
+        with f:
+            return f.readline().strip()
+
+# Import and return the group type based on string name
+def import_group_type(group_type_name):
+    mod = import_module('django_auth_ldap.config')
+    try:
+        return getattr(mod, group_type_name)()
+    except:
+        return None
 
 # Server URI
 AUTH_LDAP_SERVER_URI = os.environ.get('AUTH_LDAP_SERVER_URI', '')
@@ -13,7 +32,7 @@ AUTH_LDAP_CONNECTION_OPTIONS = {
 
 # Set the DN and password for the NetBox service account.
 AUTH_LDAP_BIND_DN = os.environ.get('AUTH_LDAP_BIND_DN', '')
-AUTH_LDAP_BIND_PASSWORD = os.environ.get('AUTH_LDAP_BIND_PASSWORD', '')
+AUTH_LDAP_BIND_PASSWORD = os.environ.get('AUTH_LDAP_BIND_PASSWORD', read_secret('auth_ldap_bind_password'))
 
 # Set a string template that describes any user’s distinguished name based on the username.
 AUTH_LDAP_USER_DN_TEMPLATE = os.environ.get('AUTH_LDAP_USER_DN_TEMPLATE', None)
@@ -23,15 +42,19 @@ AUTH_LDAP_USER_DN_TEMPLATE = os.environ.get('AUTH_LDAP_USER_DN_TEMPLATE', None)
 #     ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
 LDAP_IGNORE_CERT_ERRORS = os.environ.get('LDAP_IGNORE_CERT_ERRORS', 'False').lower() == 'true'
 
-AUTH_LDAP_USER_SEARCH = LDAPSearch(os.environ.get('AUTH_LDAP_USER_SEARCH_BASEDN', ''),
-                                    ldap.SCOPE_SUBTREE,
-                                    "(sAMAccountName=%(user)s)")
+AUTH_LDAP_USER_SEARCH_BASEDN = os.environ.get('AUTH_LDAP_USER_SEARCH_BASEDN', '')
+AUTH_LDAP_USER_SEARCH_ATTR = os.environ.get('AUTH_LDAP_USER_SEARCH_ATTR', 'sAMAccountName')
+AUTH_LDAP_USER_SEARCH = LDAPSearch(AUTH_LDAP_USER_SEARCH_BASEDN,
+                                   ldap.SCOPE_SUBTREE,
+                                   "(" + AUTH_LDAP_USER_SEARCH_ATTR + "=%(user)s)")
 
 # This search ought to return all groups to which the user belongs. django_auth_ldap uses this to determine group
 # heirarchy.
-AUTH_LDAP_GROUP_SEARCH = LDAPSearch(os.environ.get('AUTH_LDAP_GROUP_SEARCH_BASEDN', ''), ldap.SCOPE_SUBTREE,
-                                    "(objectClass=group)")
-AUTH_LDAP_GROUP_TYPE = GroupOfNamesType()
+AUTH_LDAP_GROUP_SEARCH_BASEDN = os.environ.get('AUTH_LDAP_GROUP_SEARCH_BASEDN', '')
+AUTH_LDAP_GROUP_SEARCH_CLASS = os.environ.get('AUTH_LDAP_GROUP_SEARCH_CLASS', 'group')
+AUTH_LDAP_GROUP_SEARCH = LDAPSearch(AUTH_LDAP_GROUP_SEARCH_BASEDN, ldap.SCOPE_SUBTREE,
+                                    "(objectClass=" + AUTH_LDAP_GROUP_SEARCH_CLASS + ")")
+AUTH_LDAP_GROUP_TYPE = import_group_type(os.environ.get('AUTH_LDAP_GROUP_TYPE', 'GroupOfNamesType'))
 
 # Define a group required to login.
 AUTH_LDAP_REQUIRE_GROUP = os.environ.get('AUTH_LDAP_REQUIRE_GROUP_DN', '')

docker-compose.yml

@@ -5,20 +5,20 @@ services:
       context: .
       args:
         - BRANCH=${VERSION-master}
-    image: ninech/netbox:${VERSION-latest}
+    image: netboxcommunity/netbox:${VERSION-latest}
     depends_on:
     - postgres
     - redis
     - netbox-worker
     env_file: env/netbox.env
     volumes:
-    - ./startup_scripts:/opt/netbox/startup_scripts:ro
-    - ./initializers:/opt/netbox/initializers:ro
-    - ./configuration:/etc/netbox/config:ro
-    - ./reports:/etc/netbox/reports:ro
-    - netbox-nginx-config:/etc/netbox-nginx/
-    - netbox-static-files:/opt/netbox/netbox/static
-    - netbox-media-files:/opt/netbox/netbox/media
+    - ./startup_scripts:/opt/netbox/startup_scripts:z,ro
+    - ./initializers:/opt/netbox/initializers:z,ro
+    - ./configuration:/etc/netbox/config:z,ro
+    - ./reports:/etc/netbox/reports:z,ro
+    - netbox-nginx-config:/etc/netbox-nginx:z
+    - netbox-static-files:/opt/netbox/netbox/static:z
+    - netbox-media-files:/opt/netbox/netbox/media:z
   netbox-worker:
     <<: *netbox
     depends_on:

docker/docker-entrypoint.sh

@@ -39,10 +39,14 @@ if not User.objects.filter(username='${SUPERUSER_NAME}'):
     Token.objects.create(user=u, key='${SUPERUSER_API_TOKEN}')
 END
 
-for script in /opt/netbox/startup_scripts/*.py; do
-  echo "⚙️ Executing '$script'"
-  ./manage.py shell --interface python < "${script}"
-done
+if [ "$SKIP_STARTUP_SCRIPTS" == "true" ]; then
+  echo "☇ Skipping startup scripts"
+else
+  for script in /opt/netbox/startup_scripts/*.py; do
+    echo "⚙️ Executing '$script'"
+    ./manage.py shell --interface python < "${script}"
+  done
+fi
 
 # copy static files
 ./manage.py collectstatic --no-input

env/netbox.env

@@ -1,3 +1,4 @@
+CORS_ORIGIN_ALLOW_ALL=True
 DB_NAME=netbox
 DB_USER=netbox
 DB_PASSWORD=J5brHrAXFLQSif0K
@@ -15,6 +16,9 @@ NAPALM_TIMEOUT=10
 MAX_PAGE_SIZE=1000
 REDIS_HOST=redis
 REDIS_PASSWORD=H733Kdjndks81
+REDIS_DATABASE=0
+REDIS_CACHE_DATABASE=1
+REDIS_SSL=false
 SECRET_KEY=r8OwDznj!!dci#P9ghmRfdu1Ysxm0AiPeDCQhKE+N_rClfWNj
 SUPERUSER_NAME=admin
 SUPERUSER_EMAIL=admin@example.com

hooks/build

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+. hooks/common
+
+run_build

hooks/common

@@ -0,0 +1,82 @@
+#!/bin/bash
+
+ensure_jq() {
+  echo "🛠🛠🛠 Installing JQ via apt-get"
+  [ -x "$(command -v jq)" ] || ( apt-get update && apt-get install -y jq )
+}
+
+ensure_dockerfile_present() {
+  if [ "${VARIANT}" == "main" ]; then
+    DOCKERFILE="Dockerfile"
+  else
+    DOCKERFILE="Dockerfile.${VARIANT}"
+
+    # Fail fast
+    if [ ! -f "${DOCKERFILE}" ]; then
+      echo "🚨 The Dockerfile '${DOCKERFILE}' for variant '${VARIANT}' doesn't exist."
+
+      if [ -z "$DEBUG" ]; then
+        exit 1
+      else
+        echo "⚠️ Would skip this, but DEBUG is enabled."
+      fi
+    fi
+
+    if [ "${DOCKERFILE}" != "${DOCKERFILE_PATH}" ]; then
+      echo "⚠️ The specified Dockerfile '${DOCKERFILE_PATH}' does not match the expected Dockerfile '${DOCKERFILE}'."
+      echo "   This script will use '${DOCKERFILE}' and ignore '${DOCKERFILE_PATH}'."
+    fi
+  fi
+}
+
+# Passes args to the scripts
+run_build() {
+  echo "🐳🐳🐳 Building '${BUILD}' images, the '${VARIANT:-main}' variant"
+  case $BUILD in
+    release)
+      # build the latest release
+      # shellcheck disable=SC2068
+      ./build-latest.sh $@
+      ;;
+    prerelease)
+      # build the latest pre-release
+      # shellcheck disable=SC2068
+      PRERELEASE=true ./build-latest.sh $@
+      ;;
+    branches)
+      # build all branches
+      # shellcheck disable=SC2068
+      ./build-branches.sh $@
+      ;;
+    special)
+      # special build
+      # shellcheck disable=SC2068
+      #SRC_ORG=lampwins TAG=webhooks-backend ./build.sh "feature/webhooks-backend" $@
+      echo "✅ No special builds today."
+      ;;
+    *)
+      echo "🚨 Unrecognized build '$BUILD'."
+
+      if [ -z "$DEBUG" ]; then
+        exit 1
+      else
+        echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
+      fi
+      ;;
+  esac
+}
+
+echo "🤖🤖🤖 Preparing build"
+export DOCKER_ORG="index.docker.io/netboxcommunity"
+export DOCKER_REPO=netbox
+export DOCKERHUB_REPO=netboxcommunity/netbox
+
+# mis-using the "${DOCKER_TAG}" variable as "branch to build"
+export BUILD="${DOCKER_TAG%-*}"
+export VARIANT="${DOCKER_TAG#*-}"
+
+unset DOCKER_TAG
+
+ensure_dockerfile_present
+
+ensure_jq

hooks/push

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+. hooks/common
+
+run_build --push-only

hooks/test

@@ -0,0 +1,12 @@
+#!/bin/bash
+
+. hooks/common
+
+if [ "${VARIANT}" == "main" ] && [ "${BUILD}" == "BRANCHES" ]; then
+  echo "🐳🐳🐳 Testing"
+  docker-compose pull --parallel
+  docker-compose build
+  docker-compose run netbox ./manage.py test
+else
+  echo "🐳🐳🐳 No tests are implemented for build '${BUILD}' with variant '${VARIANT}'."
+fi

startup_scripts/000_users.py

@@ -29,6 +29,6 @@ with file.open('r') as stream:
         if user_permissions:
           user.user_permissions.clear()
           for permission_codename in user_details.get('permissions', []):
-            permission = Permission.objects.get(codename=permission_codename)
-            user.user_permissions.add(permission)
+            for permission in Permission.objects.filter(codename=permission_codename):
+              user.user_permissions.add(permission)
           user.save()

startup_scripts/010_groups.py

@@ -29,5 +29,5 @@ with file.open('r') as stream:
         group.permissions.clear()
         print("Permissions:", group.permissions.all())
         for permission_codename in group_details.get('permissions', []):
-          permission = Permission.objects.get(codename=permission_codename)
-          group.permissions.add(permission)
+          for permission in Permission.objects.filter(codename=permission_codename):
+            group.permissions.add(permission)