Merge pull request #232 from netbox-community/develop

Release 0.21.1

.dockerignore

@@ -1,4 +1,11 @@
+.git
 .github
 .travis.yml
+*.md
+env
 build*
-*.env
+docker-compose.override.yml
+.netbox/.git*
+.netbox/.travis.yml
+.netbox/docs
+.netbox/scripts

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,77 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+<!--
+
+Before raising an issue here, answer the following questions for yourself, please:
+
+* Did you read through the troubleshooting section? (https://github.com/netbox-community/netbox-docker/wiki/Troubleshooting)
+* Have you had a look at the rest of the wiki? (https://github.com/netbox-community/netbox-docker/wiki)
+* Have you updated to the latest version and tried again? (i.e. `git pull` and `docker-compose pull`)
+* Have you reset the project and tried again? (i.e. `docker-compose down -v`)
+* Are you confident that your problem is related to the Docker image or Docker Compose file this project provides?
+  (Otherwise ask on the Netbox mailing list, please: https://groups.google.com/d/forum/netbox-discuss)
+* Have you looked through the issues already resolved?
+
+Please try this means to get help before opening an issue here:
+
+* On the networktocode Slack in the #netbox-docker channel: http://slack.networktocode.com/
+* On the networktocode Slack in the #netbox channel: http://slack.networktocode.com/
+* On the Netbox mailing list: https://groups.google.com/d/forum/netbox-discuss
+
+Please don't open an issue when you have a PR ready. Just submit the PR, that's good enough.
+
+-->
+
+## Current Behavior
+
+<!-- describe what you did and how it misbehaved -->
+...
+
+## Expected Behavior
+
+<!-- describe what you expected instead -->
+...
+
+## Debug Information
+
+<!-- please fill in the following information that might helps us debug your problem more quickly -->
+The output of `docker-compose version`: `XXXXX`
+The output of `docker version`: `XXXXX`
+The output of `git rev-parse HEAD`: `XXXXX`
+The command you used to start the project: `XXXXX`
+
+<!-- adjust the `latest` tag to the version you're using -->
+The output of `docker inspect netboxcommunity/netbox:latest --format "{{json .Config.Labels}}"`:
+
+```json
+{
+  "JSON JSON JSON":
+    "--> Please paste formatted json. (Use e.g. `jq` or https://jsonformatter.curiousconcept.com/)"
+}
+```
+
+The output of `docker-compose logs netbox`:
+<!--
+If your log is very long, create a Gist instead (and post the link to it): https://gist.github.com
+-->
+
+```text
+LOG LOG LOG
+```
+
+The output of `docker-compose logs nginx`:
+<!--
+Only if you have gotten a 5xx http error, else delete this section.
+If your log is very long, create a Gist instead (and post the link to it): https://gist.github.com
+-->
+
+```text
+LOG LOG LOG
+```

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,54 @@
+---
+name: Feature or Change Request
+about: Request a new feature or a change of the current behavior
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+<!--
+
+Before raising an issue here, answer the following questions for yourself, please:
+
+* Did you read through the troubleshooting section? (https://github.com/netbox-community/netbox-docker/wiki/Troubleshooting)
+* Have you had a look at the rest of the wiki? (https://github.com/netbox-community/netbox-docker/wiki)
+* Have you read the release notes recently (https://github.com/netbox-community/netbox-docker/releases)
+* Are you confident that your feature/change request is related to the Docker image or Docker Compose file this project provides?
+  (Otherwise ask on the Netbox mailing list, please: https://groups.google.com/d/forum/netbox-discuss)
+* Have you looked through the issues already resolved?
+
+Please try this means to get help before opening an issue here:
+
+* On the networktocode Slack in the #netbox-docker channel: http://slack.networktocode.com/
+* On the networktocode Slack in the #netbox channel: http://slack.networktocode.com/
+* On the Netbox mailing list: https://groups.google.com/d/forum/netbox-discuss
+
+Please don't open an issue when you have a PR ready. Just submit the PR, that's good enough.
+
+-->
+
+## Desired Behavior
+
+<!-- please describe the behavior you desire -->
+...
+
+## Contrast to Current Behavior
+
+<!-- please describe how the desired behavior is different from the current behavior -->
+...
+
+## Changes Required
+
+<!-- if you can, please elaborate what changes would exactly be required -->
+...
+
+## Discussion: Benefits and Drawbacks
+
+<!--
+Please make your case here:
+- Why do you think this project and the community will benefit from your suggestion?
+- What are the drawbacks of this change? Is it backwards-compatible?
+- Anything else that you think is relevant to the discussion of this feature/change request.
+-->
+...

.github/issue_template.md

@@ -1,47 +0,0 @@
-<!--
-
-Before raising an issue here, answer the following questions for yourself, please:
-
-* Did you read through the troubleshooting section? (https://github.com/ninech/netbox-docker/#troubleshooting)
-* Have you updated to the latest version and tried again? (i.e. `git pull` and `docker-compose pull`)
-* Have you reset the project and tried again? (i.e. `docker-compose down -v`)
-* Are you confident that your problem is related to the Docker or Docker Compose setup this project provides?
-  (Otherwise ask on the Netbox mailing list, please: https://groups.google.com/d/forum/netbox-discuss)
-* Have you looked through the issues already resolved?
-
--->
-
-## Current Behavior
-
-<!-- describe what you did and how it misbehaved -->
-...
-
-## Expected Behavior
-
-<!-- describe what you expected instead -->
-...
-
-## Debug Information
-
-<!-- please fill in the following information that might helps us debug your problem more quickly -->
-The output of `docker-compose version`: `XXXXX`
-The output of `docker version`: `XXXXX`
-The output of `git rev-parse HEAD`: `XXXXX`
-The command you used to start the project: `XXXXX`
-
-The output of `docker-compose logs netbox`:
-<!-- if your log is very long, create a Gist instead: https://gist.github.com -->
-
-```
-LOG LOG LOG
-```
-
-<!--
-If you have get any 5xx http error, else delete this section.
-If your log is very long, create a Gist instead: https://gist.github.com
--->
-The output of `docker-compose logs nginx`:
-
-```
-LOG LOG LOG
-```

.github/pull_request_template.md

@@ -0,0 +1,85 @@
+<!--
+#########################################################################
+
+Thank you for sharing your work and for opening a PR.
+
+(!) IMPORTANT (!):
+First make sure that you point your PR to the `develop` branch!
+
+Now please read the comments carefully and try to provide information
+on all relevant titles.
+
+#########################################################################
+-->
+
+<!--
+Please don't open an extra issue when submitting a PR.
+
+But if there is already a related issue, please put it's number here.
+
+E.g. #123 or N/A
+-->
+
+Related Issue:
+
+## New Behavior
+
+<!--
+Please describe in a few words the intentions of your PR.
+-->
+
+...
+
+## Contrast to Current Behavior
+
+<!--
+Please describe in a few words how the new behavior is different
+from the current behavior.
+-->
+
+...
+
+## Discussion: Benefits and Drawbacks
+
+<!--
+Please make your case here:
+
+- Why do you think this project and the community will benefit from your
+  proposed change?
+- What are the drawbacks of this change?
+- Is it backwards-compatible?
+- Anything else that you think is relevant to the discussion of this PR.
+
+(No need to write a huge article here. Just a few sentences that give some
+additional context about the motivations for the change.)
+-->
+
+...
+
+## Changes to the Wiki
+
+<!--
+If the README.md must be updated, please include the changes in the PR.
+If the Wiki must be updated, please make a suggestion below.
+-->
+
+...
+
+## Proposed Release Note Entry
+
+<!--
+Please provide a short summary of your PR that we can copy & paste
+into the release notes.
+-->
+
+...
+
+## Double Check
+
+<!--
+Please put an x into the brackets (like `[x]`) if you've completed that task.
+-->
+
+* [ ] I have read the comments and followed the PR template.
+* [ ] I have explained my PR according to the information in the comments.
+* [ ] My PR targets the `develop` branch.

.github/workflows/push.yml

@@ -0,0 +1,37 @@
+name: push
+
+on:
+  push:
+    branches-ignore:
+      - release
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        build_cmd:
+        - ./build-latest.sh
+        - PRERELEASE=true ./build-latest.sh
+        - ./build-branches.sh
+        docker_from:
+        - '' # use the default of the DOCKERFILE
+        - python:3.7-alpine
+        - python:3.8-alpine
+        # - python:3.9-rc-alpine # disable until Netbox's unit tests work
+      fail-fast: false
+    runs-on: ubuntu-latest
+    name: Builds new Netbox Docker Images
+    steps:
+    - id: git-checkout
+      name: Checkout
+      uses: actions/checkout@v1
+    - id: docker-build
+      name: Build the image from '${{ matrix.docker_from }}' with '${{ matrix.build_cmd }}'
+      run: ${{ matrix.build_cmd }}
+      env:
+        DOCKER_FROM: ${{ matrix.docker_from }}
+        GH_ACTION: enable
+    - id: docker-test
+      name: Test the image
+      run: IMAGE="${FINAL_DOCKER_TAG}" ./test.sh
+      if: steps.docker-build.outputs.skipped != 'true'

.github/workflows/release.yml

@@ -0,0 +1,51 @@
+name: release
+
+on:
+  push:
+    branches:
+      - release
+  schedule:
+    - cron: '45 5 * * *'
+
+jobs:
+  build:
+    strategy:
+      matrix:
+        build_cmd:
+        - ./build-latest.sh
+        - PRERELEASE=true ./build-latest.sh
+        - ./build-branches.sh
+      fail-fast: false
+    runs-on: ubuntu-latest
+    name: Builds new Netbox Docker Images
+    steps:
+    - id: git-checkout
+      name: Checkout
+      uses: actions/checkout@v1
+    - id: docker-build
+      name: Build the image with '${{ matrix.build_cmd }}'
+      run: ${{ matrix.build_cmd }}
+      env:
+        GH_ACTION: enable
+    - id: docker-test
+      name: Test the image
+      run: IMAGE="${FINAL_DOCKER_TAG}" ./test.sh
+      if: steps.docker-build.outputs.skipped != 'true'
+    - id: registry-login
+      name: Login to the Docker Registry
+      run: |
+        echo "::add-mask::$DOCKERHUB_USERNAME"
+        echo "::add-mask::$DOCKERHUB_PASSWORD"
+        docker login -u "$DOCKERHUB_USERNAME" --password "${DOCKERHUB_PASSWORD}" "${DOCKER_REGISTRY}"
+      env:
+        DOCKERHUB_USERNAME: ${{ secrets.dockerhub_username }}
+        DOCKERHUB_PASSWORD: ${{ secrets.dockerhub_password }}
+      if: steps.docker-build.outputs.skipped != 'true'
+    - id: registry-push
+      name: Push the image
+      run: ${{ matrix.build_cmd }} --push-only
+      if: steps.docker-build.outputs.skipped != 'true'
+    - id: registry-logout
+      name: Logout of the Docker Registry
+      run: docker logout "${DOCKER_REGISTRY}"
+      if: steps.docker-build.outputs.skipped != 'true'

.gitignore

@@ -1 +1,4 @@
 *.sql.gz
+.netbox
+.initializers
+docker-compose.override.yml

Dockerfile

@@ -1,4 +1,5 @@
-FROM python:3.6-alpine3.7
+ARG FROM=python:3.7-alpine
+FROM ${FROM} as builder
 
 RUN apk add --no-cache \
       bash \
@@ -7,53 +8,118 @@ RUN apk add --no-cache \
       cyrus-sasl-dev \
       graphviz \
       jpeg-dev \
+      libevent-dev \
       libffi-dev \
-      libxml2-dev \
       libxslt-dev \
       openldap-dev \
-      postgresql-dev \
-      ttf-ubuntu-font-family \
-      wget
+      postgresql-dev
 
-RUN pip install \
+WORKDIR /install
+
+RUN pip install --prefix="/install" --no-warn-script-location \
 # gunicorn is used for launching netbox
       gunicorn \
+      greenlet \
+      eventlet \
 # napalm is used for gathering information from network devices
       napalm \
 # ruamel is used in startup_scripts
-      ruamel.yaml \
-# pinning django to the version required by netbox
-# adding it here, to install the correct version of
-# django-rq
-      'Django>=1.11,<2.1' \
-# django-rq is used for webhooks
-      django-rq
+      'ruamel.yaml>=0.15,<0.16' \
+# django_auth_ldap is required for ldap
+      django_auth_ldap \
+# django-storages was introduced in 2.7 and is optional
+      django-storages
+
+ARG NETBOX_PATH
+COPY ${NETBOX_PATH}/requirements.txt /
+RUN pip install --prefix="/install" --no-warn-script-location -r /requirements.txt
+
+###
+# Main stage
+###
+
+ARG FROM
+FROM ${FROM} as main
+
+RUN apk add --no-cache \
+      bash \
+      ca-certificates \
+      graphviz \
+      libevent \
+      libffi \
+      libjpeg-turbo \
+      libressl \
+      libxslt \
+      postgresql-libs \
+      ttf-ubuntu-font-family
 
 WORKDIR /opt
 
-ARG BRANCH=master
-ARG URL=https://github.com/digitalocean/netbox/archive/$BRANCH.tar.gz
-RUN wget -q -O - "${URL}" | tar xz \
-  && mv netbox* netbox
+COPY --from=builder /install /usr/local
 
-WORKDIR /opt/netbox
-RUN pip install -r requirements.txt
+ARG NETBOX_PATH
+COPY ${NETBOX_PATH} /opt/netbox
 
 COPY docker/configuration.docker.py /opt/netbox/netbox/netbox/configuration.py
 COPY configuration/gunicorn_config.py /etc/netbox/config/
 COPY docker/nginx.conf /etc/netbox-nginx/nginx.conf
-COPY docker/docker-entrypoint.sh docker-entrypoint.sh
+COPY docker/docker-entrypoint.sh /opt/netbox/docker-entrypoint.sh
 COPY startup_scripts/ /opt/netbox/startup_scripts/
 COPY initializers/ /opt/netbox/initializers/
 COPY configuration/configuration.py /etc/netbox/config/configuration.py
 
 WORKDIR /opt/netbox/netbox
 
+# Must set permissions for '/opt/netbox/netbox/static' directory
+# to g+w so that `./manage.py collectstatic` can be executed during
+# container startup.
+# Must set permissions for '/opt/netbox/netbox/media' directory
+# to g+w so that pictures can be uploaded to netbox.
+RUN mkdir static && chmod g+w static media
+
 ENTRYPOINT [ "/opt/netbox/docker-entrypoint.sh" ]
 
 CMD ["gunicorn", "-c /etc/netbox/config/gunicorn_config.py", "netbox.wsgi"]
 
-LABEL SRC_URL="$URL"
+LABEL ORIGINAL_TAG="" \
+      NETBOX_GIT_BRANCH="" \
+      NETBOX_GIT_REF="" \
+      NETBOX_GIT_URL="" \
+# See http://label-schema.org/rc1/#build-time-labels
+# Also https://microbadger.com/labels
+      org.label-schema.schema-version="1.0" \
+      org.label-schema.build-date="" \
+      org.label-schema.name="Netbox Docker" \
+      org.label-schema.description="A container based distribution of Netbox, the free and open IPAM and DCIM solution." \
+      org.label-schema.vendor="The netbox-docker contributors." \
+      org.label-schema.url="https://github.com/netbox-community/netbox-docker" \
+      org.label-schema.usage="https://github.com/netbox-community/netbox-docker/wiki" \
+      org.label-schema.vcs-url="https://github.com/netbox-community/netbox-docker.git" \
+      org.label-schema.vcs-ref="" \
+      org.label-schema.version="snapshot" \
+# See https://github.com/opencontainers/image-spec/blob/master/annotations.md#pre-defined-annotation-keys
+      org.opencontainers.image.created="" \
+      org.opencontainers.image.title="Netbox Docker" \
+      org.opencontainers.image.description="A container based distribution of Netbox, the free and open IPAM and DCIM solution." \
+      org.opencontainers.image.licenses="Apache-2.0" \
+      org.opencontainers.image.authors="The netbox-docker contributors." \
+      org.opencontainers.image.vendor="The netbox-docker contributors." \
+      org.opencontainers.image.url="https://github.com/netbox-community/netbox-docker" \
+      org.opencontainers.image.documentation="https://github.com/netbox-community/netbox-docker/wiki" \
+      org.opencontainers.image.source="https://github.com/netbox-community/netbox-docker.git" \
+      org.opencontainers.image.revision="" \
+      org.opencontainers.image.version="snapshot"
 
-ARG NETBOX_DOCKER_PROJECT_VERSION=snapshot
-LABEL NETBOX_DOCKER_PROJECT_VERSION="$NETBOX_DOCKER_PROJECT_VERSION"
+#####
+## LDAP specific configuration
+#####
+
+FROM main as ldap
+
+RUN apk add --no-cache \
+      libsasl \
+      libldap \
+      util-linux
+
+COPY docker/ldap_config.docker.py /opt/netbox/netbox/netbox/ldap_config.py
+COPY configuration/ldap_config.py /etc/netbox/config/ldap_config.py

Dockerfile.ldap

@@ -1,9 +0,0 @@
-ARG DOCKER_ORG=ninech
-ARG DOCKER_REPO=netbox
-ARG FROM_TAG=latest
-FROM $DOCKER_ORG/$DOCKER_REPO:$FROM_TAG
-
-RUN pip install django_auth_ldap
-
-COPY docker/ldap_config.docker.py /opt/netbox/netbox/netbox/ldap_config.py
-COPY configuration/ldap_config.py /etc/netbox/config/ldap_config.py

README.md

@@ -1,28 +1,61 @@
 # netbox-docker
 
-[![Build Status](https://travis-ci.org/ninech/netbox-docker.svg?branch=master)][travis]
+[![GitHub release (latest by date)](https://img.shields.io/github/v/release/netbox-community/netbox-docker)][github-release]
+[![GitHub stars](https://img.shields.io/github/stars/netbox-community/netbox-docker)][github-stargazers]
+![GitHub closed pull requests](https://img.shields.io/github/issues-pr-closed-raw/netbox-community/netbox-docker)
+![Github release workflow](https://img.shields.io/github/workflow/status/netbox-community/netbox-docker/release)
+![Docker Pulls](https://img.shields.io/docker/pulls/netboxcommunity/netbox)
+[![MicroBadger Layers](https://img.shields.io/microbadger/layers/netboxcommunity/netbox)][netbox-docker-microbadger]
+[![MicroBadger Size](https://img.shields.io/microbadger/image-size/netboxcommunity/netbox)][netbox-docker-microbadger]
+[![GitHub license](https://img.shields.io/github/license/netbox-community/netbox-docker)][netbox-docker-license]
 
-This repository houses the components needed to build NetBox as a Docker container.
-Images built using this code are released to [Docker Hub][netbox-dockerhub] every night.
+[The Github repository](netbox-docker-github) houses the components needed to build Netbox as a Docker container.
+Images are built using this code and are released to [Docker Hub][netbox-dockerhub] once a day.
 
-[travis]: https://travis-ci.org/ninech/netbox-docker
-[netbox-dockerhub]: https://hub.docker.com/r/ninech/netbox/tags/
+Do you have any questions? Before opening an issue on Github, please join the [Network To Code][ntc-slack] Slack and ask for help in our [`#netbox-docker`][netbox-docker-slack] channel.
+
+[github-stargazers]: https://github.com/netbox-community/netbox-docker/stargazers
+[github-release]: https://github.com/netbox-community/netbox-docker/releases
+[netbox-docker-microbadger]: https://microbadger.com/images/netboxcommunity/netbox
+[netbox-dockerhub]: https://hub.docker.com/r/netboxcommunity/netbox/tags/
+[netbox-docker-github]: https://github.com/netbox-community/netbox-docker/
+[ntc-slack]: http://slack.networktocode.com/
+[netbox-docker-slack]: https://slack.com/app_redirect?channel=netbox-docker&team=T09LQ7E9E
+[netbox-docker-license]: https://github.com/netbox-community/netbox-docker/blob/release/LICENSE
+
+## Docker Tags
+
+* `vX.Y.Z`: Release builds, built from [releases of Netbox][netbox-releases].
+* `latest`: Release builds, built from [`master` branch of Netbox][netbox-master].
+* `snapshot`: Pre-release builds, built from the [`develop` branch of Netbox][netbox-develop].
+* `develop-X.Y`: Pre-release builds, built from the corresponding [branch of Netbox][netbox-branches].
+
+Then there is currently one extra tags for each of the above labels:
+
+* `-ldap`: Contains additional dependencies and configurations for connecting Netbox to an LDAP directroy.
+  [Learn more about that in our wiki][netbox-docker-ldap].
+
+[netbox-releases]: https://github.com/netbox-community/netbox/releases
+[netbox-master]: https://github.com/netbox-community/netbox/tree/master
+[netbox-develop]: https://github.com/netbox-community/netbox/tree/develop
+[netbox-branches]: https://github.com/netbox-community/netbox/branches
+[netbox-docker-ldap]: https://github.com/netbox-community/netbox-docker/wiki/LDAP
 
 ## Quickstart
 
-To get NetBox up and running:
+To get Netbox up and running in Docker:
 
-```
-$ git clone -b master https://github.com/ninech/netbox-docker.git
-$ cd netbox-docker
-$ docker-compose pull
-$ docker-compose up -d
+```bash
+git clone -b release https://github.com/netbox-community/netbox-docker.git
+cd netbox-docker
+docker-compose pull
+docker-compose up -d
 ```
 
 The application will be available after a few minutes.
 Use `docker-compose port nginx 8080` to find out where to connect to.
 
-```
+```bash
 $ echo "http://$(docker-compose port nginx 8080)/"
 http://0.0.0.0:32768/
 
@@ -35,371 +68,94 @@ $ xdg-open "http://$(docker-compose port nginx 8080)/" &>/dev/null &
 
 Alternatively, use something like [Reception][docker-reception] to connect to _docker-compose_ projects.
 
-Default credentials:
+The default credentials are:
 
 * Username: **admin**
 * Password: **admin**
 * API Token: **0123456789abcdef0123456789abcdef01234567**
 
-[docker-reception]: https://github.com/ninech/reception
+There is a more complete [Getting Started guide on our Wiki][wiki-getting-started].
+
+[wiki-getting-started]: https://github.com/netbox-community/netbox-docker/wiki/Getting-Started
+[docker-reception]: https://github.com/nxt-engineering/reception
 
 ## Dependencies
 
-This project relies only on *Docker* and *docker-compose* meeting this requirements:
+This project relies only on *Docker* and *docker-compose* meeting these requirements:
 
-* The *Docker version* must be at least `1.13.0`.
-* The *docker-compose version* must be at least `1.10.0`.
+* The *Docker version* must be at least `17.05`.
+* The *docker-compose version* must be at least `1.17.0`.
 
-To ensure this, compare the output of `docker --version` and `docker-compose --version` with the requirements above.
+To check the version installed on your system run `docker --version` and `docker-compose --version`.
 
-## Configuration
+## Documentation
 
-You can configure the app using environment variables.
-These are defined in `netbox.env`.
-Read [Environment Variables in Compose][compose-env] to understand about the various possibilities to overwrite these variables.
-(The easiest solution being simply adjusting that file.)
+Please refer [to our wiki on Github][netbox-docker-wiki] for further information on how to use this Netbox Docker image properly.
+It covers advanced topics such as using secret files, deployment to Kubernetes as well as NAPALM and LDAP configuration.
 
-To find all possible variables, have a look at the [configuration.docker.py][docker-config] and [docker-entrypoint.sh][entrypoint] files.
-Generally, the environment variables are called the same as their respective NetBox configuration variables.
-Variables which are arrays are usually composed by putting all the values into the same environment variables with the values separated by a whitespace ("` `").
-For example defining `ALLOWED_HOSTS=localhost ::1 127.0.0.1` would allows access to NetBox through `http://localhost:8080`, `http://[::1]:8080` and `http://127.0.0.1:8080`.
-
-[compose-env]: https://docs.docker.com/compose/environment-variables/
-
-### Production
-
-The default settings are optimized for (local) development environments.
-You should therefore adjust the configuration for production setups, at least the following variables:
-
-* `ALLOWED_HOSTS`: Add all URLs that lead to your NetBox instance, space separated. E.g. `ALLOWED_HOSTS=netbox.mycorp.com server042.mycorp.com 2a02:123::42 10.0.0.42 localhost ::1 127.0.0.1` (It's good advice to always allow localhost connections for easy debugging, i.e. `localhost ::1 127.0.0.1`.)
-* `DB_*`: Use your own persistent database. Don't use the default passwords!
-* `EMAIL_*`: Use your own mailserver.
-* `MAX_PAGE_SIZE`: Use the recommended default of 1000.
-* `SUPERUSER_*`: Only define those variables during the initial setup, and drop them once the DB is set up. Don't use the default passwords!
-* `REDIS_*`: Use your own persistent redis. Don't use the default passwords!
-
-### Running on Docker Swarm / Kubernetes / OpenShift
-
-You may run this image in a cluster such as Docker Swarm, Kubernetes or OpenShift, but this is advanced level.
-
-In this case, we encourage you to statically configure NetBox by starting from [NetBox's example config file][default-config], and mounting it into your container in the directory `/etc/netbox/config/` using the mechanism provided by your container platform (i.e. [Docker Swarm configs][swarm-config], [Kubernetes ConfigMap][k8s-config], [OpenShift ConfigMaps][openshift-config]).
-
-But if you rather continue to configure your application through environment variables, you may continue to use [the built-in configuration file][docker-config].
-We discourage storing secrets in environment variables, as environment variable are passed on to all sub-processes and may leak easily into other systems, e.g. error collecting tools that often collect all environment variables whenever an error occurs.
-
-Therefore we *strongly advise* to make use of the secrets mechanism provided by your container platform (i.e. [Docker Swarm secrets][swarm-secrets], [Kubernetes secrets][k8s-secrets], [OpenShift secrets][openshift-secrets]).
-[The configuration file][docker-config] and [the entrypoint script][entrypoint] try to load the following secrets from the respective files.
-If a secret is defined by an environment variable and in the respective file at the same time, then the value from the environment variable is used.
-
-* `SUPERUSER_PASSWORD`: `/run/secrets/superuser_password`
-* `SUPERUSER_API_TOKEN`: `/run/secrets/superuser_api_token`
-* `DB_PASSWORD`: `/run/secrets/db_password`
-* `SECRET_KEY`: `/run/secrets/secret_key`
-* `EMAIL_PASSWORD`: `/run/secrets/email_password`
-* `NAPALM_PASSWORD`: `/run/secrets/napalm_password`
-* `REDIS_PASSWORD`: `/run/secrets/redis_password`
-
-Please also consider [the advice about running NetBox in production](#production) above!
-
-[docker-config]: https://github.com/ninech/netbox-docker/blob/master/docker/configuration.docker.py
-[default-config]: https://github.com/digitalocean/netbox/blob/develop/netbox/netbox/configuration.example.py
-[entrypoint]: https://github.com/ninech/netbox-docker/blob/master/docker/docker-entrypoint.sh
-[swarm-config]: https://docs.docker.com/engine/swarm/configs/
-[swarm-secrets]: https://docs.docker.com/engine/swarm/secrets/
-[openshift-config]: https://docs.openshift.org/latest/dev_guide/configmaps.html
-[openshift-secrets]: https://docs.openshift.org/latest/dev_guide/secrets.html
-[k8s-secrets]: https://kubernetes.io/docs/concepts/configuration/secret/
-[k8s-config]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/
-
-### NAPALM Configuration
-
-Since v2.1.0 NAPALM has been tightly integrated into NetBox.
-NAPALM allows NetBox to fetch live data from devices and return it to a requester via its REST API.
-To learn more about what NAPALM is and how it works, please see the documentation from the [libary itself][napalm-doc] or the documentation from [NetBox][netbox-napalm-doc] on how it is integrated.
-
-To enable this functionality, simply complete the following lines in `netbox.env` (or appropriate secrets mechanism) :
-
-* `NAPALM_USERNAME`: A common username that can be utilized for connecting to network devices in your environment.
-* `NAPALM_PASSWORD`: The password to use in combintation with the username to connect to network devices.
-* `NAPALM_TIMEOUT`: A value to use for when an attempt to connect to a device will timeout if no response has been recieved.
-
-However, if you don't need this functionality, leave these blank.
-
-[napalm-doc]: http://napalm.readthedocs.io/en/latest/index.html
-[netbox-napalm-doc]: https://netbox.readthedocs.io/en/latest/configuration/optional-settings/#napalm_username
-
-### Custom Initialization Code (e.g. Automatically Setting Up Custom Fields)
-
-When using `docker-compose`, all the python scripts present in `/opt/netbox/startup_scripts` will automatically be executed after the application boots in the context of `./manage.py`.
-
-That mechanism can be used for many things, e.g. to create NetBox custom fields:
-
-```python
-# docker/startup_scripts/load_custom_fields.py
-from django.contrib.contenttypes.models import ContentType
-from extras.models import CF_TYPE_TEXT, CustomField
-
-from dcim.models import Device
-from dcim.models import DeviceType
-
-device      = ContentType.objects.get_for_model(Device)
-device_type = ContentType.objects.get_for_model(DeviceType)
-
-my_custom_field, created = CustomField.objects.get_or_create(
-    type=CF_TYPE_TEXT,
-    name='my_custom_field',
-    description='My own custom field'
-)
-
-if created:
-  my_custom_field.obj_type.add(device)
-  my_custom_field.obj_type.add(device_type)
-```
-
-#### Initializers
-
-Initializers are built-in startup scripts for defining NetBox custom fields, groups and users.
-All you need to do is to mount you own `initializers` folder ([see `docker-compose.yml`][netbox-docker-compose]).
-Look at the [`initializers` folder][netbox-docker-initializers] to learn how the files must look like.
-
-Here's an example for defining a custom field:
-
-```yaml
-# initializers/custom_fields.yml
-text_field:
-  type: text
-  label: Custom Text
-  description: Enter text in a text field.
-  required: false
-  filter_logic: loose
-  weight: 0
-  on_objects:
-  - dcim.models.Device
-  - dcim.models.Rack
-  - ipam.models.IPAddress
-  - ipam.models.Prefix
-  - tenancy.models.Tenant
-  - virtualization.models.VirtualMachine
-```
-
-[netbox-docker-initializers]: https://github.com/ninech/netbox-docker/tree/master/initializers
-[netbox-docker-compose]: https://github.com/ninech/netbox-docker/blob/master/docker-compose.yml
-
-#### Custom Docker Image
-
-You can also build your own NetBox Docker image containing your own startup scripts, custom fields, users and groups
-like this:
-
-```
-ARG VERSION=latest
-FROM ninech/netbox:$VERSION
-
-COPY startup_scripts/ /opt/netbox/startup_scripts/
-COPY initializers/ /opt/netbox/initializers/
-```
+[netbox-docker-wiki]: https://github.com/netbox-community/netbox-docker/wiki/
 
 ## Netbox Version
 
-The `docker-compose.yml` file is prepared to run a specific version of NetBox.
+The `docker-compose.yml` file is prepared to run a specific version of Netbox.
 To use this feature, set the environment-variable `VERSION` before launching `docker-compose`, as shown below.
 `VERSION` may be set to the name of
-[any tag of the `ninech/netbox` Docker image on Docker Hub][netbox-dockerhub].
+[any tag of the `netboxcommunity/netbox` Docker image on Docker Hub][netbox-dockerhub].
 
-```
-$ export VERSION=v2.2.6
-$ docker-compose pull netbox
-$ docker-compose up -d
+```bash
+export VERSION=v2.7.1
+docker-compose pull netbox
+docker-compose up -d
 ```
 
-You can also build a specific version of the NetBox image. This time, `VERSION` indicates any valid
-[Git Reference][git-ref] declared on [the 'digitalocean/netbox' Github repository][netbox-github].
-Most commonly you will specify a tag or branch name.
+You can also build a specific version of the Netbox Docker image yourself.
+`VERSION` can be any valid [git ref][git-ref] in that case.
 
+```bash
+export VERSION=v2.7.1
+./build.sh $VERSION
+docker-compose up -d
 ```
-$ export VERSION=develop
-$ docker-compose build --no-cache netbox
-$ docker-compose up -d
-```
-
-Hint: If you're building a specific version by tag name, the `--no-cache` argument is not strictly necessary.
-This can increase the build speed if you're just adjusting the config, for example.
 
 [git-ref]: https://git-scm.com/book/en/v2/Git-Internals-Git-References
-[netbox-github]: https://github.com/digitalocean/netbox/releases
+[netbox-github]: https://github.com/netbox-community/netbox/releases
 
-### LDAP enabled variant
-
-The images tagged with "-ldap" contain anything necessary to authenticate against an LDAP or Active Directory server.
-The default configuration `ldap_config.py` is prepared for use with an Active Directory server.
-Custom values can be injected using environment variables, similar to the main configuration mechanisms.
-
-## Troubleshooting
-
-This section is a collection of some common issues and how to resolve them.
-If your issue is not here, look through [the existing issues][issues] and eventually create a new issue.
-
-[issues]: (https://github.com/ninech/netbox-docker/issues)
-
-### Docker Compose basics
-
-* You can see all running containers belonging to this project using `docker-compose ps`.
-* You can see the logs by running `docker-compose logs -f`.
-  Running `docker-compose logs -f netbox` will just show the logs for netbox.
-* You can stop everything using `docker-compose stop`.
-* You can clean up everything using `docker-compose down -v --remove-orphans`. **This will also remove any related data.**
-* You can enter the shell of the running NetBox container using `docker-compose exec netbox /bin/bash`. Now you have access to `./manage.py`, e.g. to reset a password.
-* To access the database run `docker-compose exec postgres sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'`
-* To create a database backup run `docker-compose exec postgres sh -c 'pg_dump -cU $POSTGRES_USER $POSTGRES_DB' | gzip > db_dump.sql.gz`
-* To restore that database backup run `gunzip -c db_dump.sql.gz | docker exec -i $(docker-compose ps -q postgres) sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'`.
-
-### Nginx doesn't start
-
-As a first step, stop your docker-compose setup.
-Then locate the `netbox-nginx-config` volume and remove it:
-
-```bash
-# Stop your local netbox-docker installation
-$ docker-compose down
-
-# Find the volume
-$ docker volume ls | grep netbox-nginx-config
-local               netbox-docker_netbox-nginx-config
-
-# Remove the volume
-$ docker volume rm netbox-docker_netbox-nginx-config
-netbox-docker_netbox-nginx-config
-```
-
-Now start everything up again.
-
-If this didn't help, try to see if there's anything in the logs indicating why nginx doesn't start:
-
-```bash
-$ docker-compose logs -f nginx
-```
-
-### Getting a "Bad Request (400)"
-
-> When connecting to the NetBox instance, I get a "Bad Request (400)" error.
-
-This usually happens when the `ALLOWED_HOSTS` variable is not set correctly.
-
-### How to upgrade
-
-> How do I update to a newer version of netbox?
-
-It should be sufficient to pull the latest image from Docker Hub, stopping the container and starting it up again:
-
-```bash
-docker-compose pull netbox
-docker-compose stop netbox netbox-worker
-docker-compose rm -f netbox netbox-worker
-docker-compose up -d netbox netbox-worker
-```
-
-### Webhooks don't work
-
-First make sure that the webhooks feature is enabled in your Netbox configuration and that a redis host is defined.
-Check `netbox.env` if the following variables are defined:
-
-```
-WEBHOOKS_ENABLED=true
-REDIS_HOST=redis
-```
-
-Then make sure that the `redis` container and at least one `netbox-worker` are running.
-
-```
-# check the container status
-$ docker-compose ps
-
-Name                           Command               State                Ports             
---------------------------------------------------------------------------------------------------------
-netbox-docker_netbox-worker_1   /opt/netbox/docker-entrypo ...   Up                                     
-netbox-docker_netbox_1          /opt/netbox/docker-entrypo ...   Up                                     
-netbox-docker_nginx_1           nginx -c /etc/netbox-nginx ...   Up      80/tcp, 0.0.0.0:32776->8080/tcp
-netbox-docker_postgres_1        docker-entrypoint.sh postgres    Up      5432/tcp
-netbox-docker_redis_1           docker-entrypoint.sh redis ...   Up      6379/tcp
-
-# connect to redis and send PING command:
-$ docker-compose run --rm -T redis sh -c 'redis-cli -h redis -a $REDIS_PASSWORD ping'
-Warning: Using a password with '-a' option on the command line interface may not be safe.
-PONG
-```
-
-If `redis` and the `netbox-worker` are not available, make sure you have updated your `docker-compose.yml` file!
-
-Everything's up and running? Then check the log of `netbox-worker` and/or `redis`:
-
-```bash
-docker-compose logs -f netbox-worker
-docker-compose logs -f redis
-```
-
-Still no clue? You can connect to the `redis` container and have it report any command that is currently executed on the server:
-
-```bash
-docker-compose run --rm -T redis sh -c 'redis-cli -h redis -a $REDIS_PASSWORD monitor'
-
-# Hit CTRL-C a few times to leave
-```
-
-If you don't see anything happening after you triggered a webhook, double-check the configuration of the `netbox` and the `netbox-worker` containers and also check the configuration of your webhook in the admin interface of Netbox.
-
-### Breaking Changes
+## Breaking Changes
 
 From time to time it might become necessary to re-engineer the structure of this setup.
 Things like the `docker-compose.yml` file or your Kubernetes or OpenShift configurations have to be adjusted as a consequence.
-Since April 2018 each image built from this repo contains a `NETBOX_DOCKER_PROJECT_VERSION` label.
-You can check the label of your local image by running `docker inspect ninech/netbox:v2.3.1 --format "{{json .ContainerConfig.Labels}}"`.
-Compare the version with the list below to check whether a breaking change was introduced with that version.
 
-The following is a list of breaking changes of the `netbox-docker` project:
+Since November 2019 each image built from this repo contains a `org.opencontainers.image.version` label.
+(The images contained labels since April 2018, although in November 2019 the labels' names changed.)
+You can check the label of your local image by running `docker inspect netboxcommunity/netbox:v2.7.1 --format "{{json .Config.Labels}}"`.
 
-* 0.4.0: In order to use Netbox webhooks you need to add Redis and a netbox-worker to your docker-compose.yml.
-* 0.3.0: Field `filterable: <boolean` was replaced with field `filter_logic: loose/exact/disabled`. It will default to `CF_FILTER_LOOSE=loose` when not defined.
-* 0.2.0: Re-organized paths: `/etc/netbox -> /etc/netbox/config` and `/etc/reports -> /etc/netbox/reports`. Fixes [#54](https://github.com/ninech/netbox-docker/issues/54).
-* 0.1.0: Introduction of the `NETBOX_DOCKER_PROJECT_VERSION`. (Not a breaking change per se.)
+Please read [the release notes][releases] carefully when updating to a new image version.
 
-## Rebuilding & Publishing images
+[releases]: https://github.com/netbox-community/netbox-docker/releases
 
-`./build.sh` is used to rebuild the Docker image:
+## Rebuilding the Image
 
-```
-$ ./build.sh --help
-Usage: ./build.sh <branch> [--push]
-  branch  The branch or tag to build. Required.
-  --push  Pushes built Docker image to docker hub.
+`./build.sh` can be used to rebuild the Docker image. See `./build.sh --help` for more information.
 
-You can use the following ENV variables to customize the build:
-  BRANCH   The branch to build.
-           Also used for tagging the image.
-  DOCKER_REPO The Docker registry (i.e. hub.docker.com/r/DOCKER_REPO/netbox)
-           Also used for tagging the image.
-           Default: ninech
-  SRC_REPO Which fork of netbox to use (i.e. github.com/<SRC_REPO>/netbox).
-           Default: digitalocean
-  URL      Where to fetch the package from.
-           Must be a tar.gz file of the source code.
-           Default: https://github.com/${SRC_REPO}/netbox/archive/$BRANCH.tar.gz
-```
+For more details on custom builds [consult our wiki][netbox-docker-wiki-build].
 
-### Publishing Docker Images
+[netbox-docker-wiki-build]: https://github.com/netbox-community/netbox-docker/wiki/Build
 
-New Docker Images are built and published every 24h by using travis:
+### Pre-made Docker Images
 
-[![Build Status](https://travis-ci.org/ninech/netbox-docker.svg?branch=master)][travis]
+New Docker images are built and published every 24h.
 
 ## Tests
 
-To run the tests coming with NetBox, use the `docker-compose.yml` file as such:
+We have a test script.
+It runs Netbox's own unit tests and ensures that all initializers work:
 
-```
-$ docker-compose run netbox ./manage.py test
+```bash
+IMAGE=netboxcommunity/netbox:latest ./test.sh
 ```
 
 ## About
 
-This repository is currently maintained and funded by [nine](https://nine.ch), your cloud navigator.
+This repository is currently maintained and funded by [nxt][nxt].
 
-[![logo of the company 'nine'](https://logo.apps.at-nine.ch/Dmqied_eSaoBMQwk3vVgn4UIgDo=/trim/500x0/logo_claim.png)](https://www.nine.ch)
+[nxt]: https://nxt.engineering/en/

VERSION

@@ -1 +1 @@
-0.4.0
+0.21.1

build-all.sh

@@ -12,77 +12,40 @@ BUILDS=("${BUILD:-"${ALL_BUILDS[@]}"}")
 
 echo "⚙️ Configured builds: ${BUILDS[*]}"
 
-VARIANTS=("" "ldap")
-
-if [ ! -z "${DEBUG}" ]; then
+if [ -n "${DEBUG}" ]; then
   export DEBUG
 fi
 
 ERROR=0
 
-# Don't build if not on `master` and don't build if on a pull request,
-# but build when DEBUG is not empty
-if [ ! -z "${DEBUG}" ] || \
-  ( [ "$TRAVIS_BRANCH" = "master" ] && [ "$TRAVIS_PULL_REQUEST" = "false" ] ); then
-  for VARIANT in "${VARIANTS[@]}"; do
-		export VARIANT
+for BUILD in "${BUILDS[@]}"; do
+  echo "🛠 Building '$BUILD' from '$DOCKERFILE'"
+  case $BUILD in
+    release)
+      # build the latest release
+      # shellcheck disable=SC2068
+      ./build-latest.sh $@ || ERROR=1
+      ;;
+    prerelease)
+      # build the latest pre-release
+      # shellcheck disable=SC2068
+      PRERELEASE=true ./build-latest.sh $@ || ERROR=1
+      ;;
+    branches)
+      # build all branches
+      # shellcheck disable=SC2068
+      ./build-branches.sh $@ || ERROR=1
+      ;;
+    *)
+      echo "🚨 Unrecognized build '$BUILD'."
 
-    # Checking which VARIANT to build
-    if [ -z "$VARIANT" ]; then
-      DOCKERFILE="Dockerfile"
-    else
-      DOCKERFILE="Dockerfile.${VARIANT}"
-
-      # Fail fast
-      if [ ! -f "${DOCKERFILE}" ]; then
-        echo "🚨 The Dockerfile '${DOCKERFILE}' for variant '${VARIANT}' doesn't exist."
-        ERROR=1
-
-        if [ -z "$DEBUG" ]; then
-          continue
-        else
-          echo "⚠️ Would skip this, but DEBUG is enabled."
-        fi
+      if [ -z "$DEBUG" ]; then
+        exit 1
+      else
+        echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
       fi
-    fi
-
-    for BUILD in "${BUILDS[@]}"; do
-      echo "🛠 Building '$BUILD' from '$DOCKERFILE'"
-      case $BUILD in
-        release)
-          # build the latest release
-          # shellcheck disable=SC2068
-          ./build-latest.sh $@ || ERROR=1
-          ;;
-        prerelease)
-          # build the latest pre-release
-          # shellcheck disable=SC2068
-      		PRERELEASE=true ./build-latest.sh $@ || ERROR=1
-          ;;
-        branches)
-          # build all branches
-          # shellcheck disable=SC2068
-      		./build-branches.sh $@ || ERROR=1
-          ;;
-        special)
-          # special build
-          # shellcheck disable=SC2068
-      		#SRC_ORG=lampwins TAG=webhooks-backend ./build.sh "feature/webhooks-backend" $@ || ERROR=1
-          ;;
-        *)
-          echo "🚨 Unrecognized build '$BUILD'."
-
-          if [ -z "$DEBUG" ]; then
-            exit 1
-          else
-            echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
-          fi
-          ;;
-      esac
-    done
-  done
-else
-  echo "❎ Not building anything."
-fi
+      ;;
+  esac
+done
 
 exit $ERROR

build-branches.sh

@@ -1,20 +1,47 @@
 #!/bin/bash
-# Builds develop, develop-* and master branches
+# Builds develop, develop-* and master branches of Netbox
 
 echo "▶️ $0 $*"
 
-ORIGINAL_GITHUB_REPO="digitalocean/netbox"
+###
+# Checking for the presence of GITHUB_OAUTH_CLIENT_ID
+# and GITHUB_OAUTH_CLIENT_SECRET
+###
+if [ -n "${GITHUB_OAUTH_CLIENT_ID}" ] && [ -n "${GITHUB_OAUTH_CLIENT_SECRET}" ]; then
+  echo "🗝 Performing authenticated Github API calls."
+  GITHUB_OAUTH_PARAMS="client_id=${GITHUB_OAUTH_CLIENT_ID}&client_secret=${GITHUB_OAUTH_CLIENT_SECRET}"
+else
+  echo "🕶 Performing unauthenticated Github API calls. This might result in lower Github rate limits!"
+  GITHUB_OAUTH_PARAMS=""
+fi
+
+###
+# Calling Github to get the all branches
+###
+ORIGINAL_GITHUB_REPO="${SRC_ORG-netbox-community}/${SRC_REPO-netbox}"
 GITHUB_REPO="${GITHUB_REPO-$ORIGINAL_GITHUB_REPO}"
-URL_RELEASES="https://api.github.com/repos/${GITHUB_REPO}/branches"
+URL_RELEASES="https://api.github.com/repos/${GITHUB_REPO}/branches?${GITHUB_OAUTH_PARAMS}"
+
+# Composing the JQ commans to extract the most recent version number
+JQ_BRANCHES='map(.name) | .[] | scan("^[^v].+") | match("^(master|develop).*") | .string'
 
 CURL="curl -sS"
 
-BRANCHES=$($CURL "${URL_RELEASES}" | jq -r 'map(.name) | .[] | scan("^[^v].+") | match("^(master|develop).*") | .string')
+# Querying the Github API to fetch all branches
+BRANCHES=$($CURL "${URL_RELEASES}" | jq -r "$JQ_BRANCHES")
 
+###
+# Building each branch
+###
+
+# keeping track whether an error occured
 ERROR=0
 
+# calling build.sh for each branch
 for BRANCH in $BRANCHES; do
   # shellcheck disable=SC2068
   ./build.sh "${BRANCH}" $@ || ERROR=1
 done
+
+# returning whether an error occured
 exit $ERROR

build-latest.sh

@@ -3,17 +3,50 @@
 
 echo "▶️ $0 $*"
 
-ORIGINAL_GITHUB_REPO="digitalocean/netbox"
-GITHUB_REPO="${GITHUB_REPO-$ORIGINAL_GITHUB_REPO}"
-URL_RELEASES="https://api.github.com/repos/${GITHUB_REPO}/releases"
+###
+# Checking for the presence of GITHUB_OAUTH_CLIENT_ID
+# and GITHUB_OAUTH_CLIENT_SECRET
+###
+if [ -n "${GITHUB_OAUTH_CLIENT_ID}" ] && [ -n "${GITHUB_OAUTH_CLIENT_SECRET}" ]; then
+  echo "🗝 Performing authenticated Github API calls."
+  GITHUB_OAUTH_PARAMS="client_id=${GITHUB_OAUTH_CLIENT_ID}&client_secret=${GITHUB_OAUTH_CLIENT_SECRET}"
+else
+  echo "🕶 Performing unauthenticated Github API calls. This might result in lower Github rate limits!"
+  GITHUB_OAUTH_PARAMS=""
+fi
 
+###
+# Checking if PRERELEASE is either unset, 'true' or 'false'
+###
+if [ -n "${PRERELEASE}" ] &&
+   { [ "${PRERELEASE}" != "true" ] && [ "${PRERELEASE}" != "false" ]; }; then
+
+  if [ -z "${DEBUG}" ]; then
+    echo "⚠️ PRERELEASE must be either unset, 'true' or 'false', but was '${PRERELEASE}'!"
+    exit 1
+  else
+    echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
+  fi
+fi
+
+###
+# Calling Github to get the latest version
+###
+ORIGINAL_GITHUB_REPO="netbox-community/netbox"
+GITHUB_REPO="${GITHUB_REPO-$ORIGINAL_GITHUB_REPO}"
+URL_RELEASES="https://api.github.com/repos/${GITHUB_REPO}/releases?${GITHUB_OAUTH_PARAMS}"
+
+# Composing the JQ commans to extract the most recent version number
 JQ_LATEST="group_by(.prerelease) | .[] | sort_by(.published_at) | reverse | .[0] | select(.prerelease==${PRERELEASE-false}) | .tag_name"
 
 CURL="curl -sS"
 
+# Querying the Github API to fetch the most recent version number
 VERSION=$($CURL "${URL_RELEASES}" | jq -r "${JQ_LATEST}")
 
+###
 # Check if the prerelease version is actually higher than stable version
+###
 if [ "${PRERELEASE}" == "true" ]; then
   JQ_STABLE="group_by(.prerelease) | .[] | sort_by(.published_at) | reverse | .[0] | select(.prerelease==false) | .tag_name"
   STABLE_VERSION=$($CURL "${URL_RELEASES}" | jq -r "${JQ_STABLE}")
@@ -27,10 +60,16 @@ if [ "${PRERELEASE}" == "true" ]; then
   # shellcheck disable=SC2003
   MINOR_UNSTABLE=$(expr match "${VERSION}" 'v[0-9]\+\.\([0-9]\+\)')
 
-  if ( [ "$MAJOR_STABLE" -eq "$MAJOR_UNSTABLE" ] && [ "$MINOR_STABLE" -ge "$MINOR_UNSTABLE" ] ) \
-     || [ "$MAJOR_STABLE" -gt "$MAJOR_UNSTABLE" ]; then
-    echo "❎ Latest unstable version ('$VERSION') is not higher than the latest stable version ('$STABLE_VERSION')."
+  if { [ "${MAJOR_STABLE}" -eq "${MAJOR_UNSTABLE}" ] \
+       && [ "${MINOR_STABLE}" -ge "${MINOR_UNSTABLE}" ];
+     } || [ "${MAJOR_STABLE}" -gt "${MAJOR_UNSTABLE}" ]; then
+
+    echo "❎ Latest unstable version '${VERSION}' is not higher than the latest stable version '$STABLE_VERSION'."
     if [ -z "$DEBUG" ]; then
+      if [ -n "${GH_ACTION}" ]; then
+        echo "::set-output name=skipped::true"
+      fi
+
       exit 0
     else
       echo "⚠️ Would exit here with code '0', but DEBUG is enabled."
@@ -38,28 +77,6 @@ if [ "${PRERELEASE}" == "true" ]; then
   fi
 fi
 
-# Check if that version is not already available on docker hub:
-ORIGINAL_DOCKERHUB_REPO="ninech/netbox"
-DOCKERHUB_REPO="${DOCKERHUB_REPO-$ORIGINAL_DOCKERHUB_REPO}"
-URL_DOCKERHUB_TOKEN="https://auth.docker.io/token?service=registry.docker.io&scope=repository:${DOCKERHUB_REPO}:pull"
-BEARER_TOKEN="$($CURL "${URL_DOCKERHUB_TOKEN}" | jq -r .token)"
-
-URL_DOCKERHUB_TAG="https://registry.hub.docker.com/v2/${DOCKERHUB_REPO}/tags/list"
-AUTHORIZATION_HEADER="Authorization: Bearer ${BEARER_TOKEN}"
-
-if [ -z "$VARIANT" ]; then
-  DOCKER_TAG="${VERSION}"
-else
-  DOCKER_TAG="${VERSION}-${VARIANT}"
-fi
-
-ALREADY_BUILT="$($CURL -H "${AUTHORIZATION_HEADER}" "${URL_DOCKERHUB_TAG}" | jq -e ".tags | any(.==\"${DOCKER_TAG}\")")"
-
-if [ "$ALREADY_BUILT" == "false" ]; then
-  # shellcheck disable=SC2068
-  ./build.sh "${VERSION}" $@
-  exit $?
-else
-  echo "✅ ${DOCKER_TAG} already exists on https://hub.docker.com/r/${DOCKERHUB_REPO}"
-  exit 0
-fi
+# shellcheck disable=SC2068
+./build.sh "${VERSION}" $@
+exit $?

build.sh

@@ -1,68 +1,92 @@
 #!/bin/bash
-# Builds the Dockerfile[.variant] and injects tgz'ed Netbox code from Github
+# Clones the Netbox repository with git from Github and builds the Dockerfile
 
 echo "▶️ $0 $*"
 
 set -e
 
 if [ "${1}x" == "x" ] || [ "${1}" == "--help" ] || [ "${1}" == "-h" ]; then
-  echo "Usage: ${0} <branch> [--push]"
-  echo "  branch  The branch or tag to build. Required."
-  echo "  --push  Pushes built Docker image to docker hub."
+  echo "Usage: ${0} <branch> [--push|--push-only]"
+  echo "  branch       The branch or tag to build. Required."
+  echo "  --push       Pushes the built Docker image to the registry."
+  echo "  --push-only  Only pushes the Docker image to the registry, but does not build it."
   echo ""
   echo "You can use the following ENV variables to customize the build:"
-  echo "  DEBUG    If defined, the script does not stop when certain checks are unsatisfied."
-  echo "  DRY_RUN  Prints all build statements instead of running them."
-  echo "  DOCKER_OPTS Add parameters to Docker."
-  echo "           Default:"
-  echo "             When <TAG> starts with 'v':  \"\""
-  echo "             Else:                        \"--no-cache\""
-  echo "  BRANCH   The branch to build."
-  echo "           Also used for tagging the image."
-  echo "  TAG      The version part of the docker tag."
-  echo "           Default:"
-  echo "             When <BRANCH>=master:  latest"
-  echo "             When <BRANCH>=develop: snapshot"
-  echo "             Else:          same as <BRANCH>"
-  echo "  DOCKER_ORG The Docker registry (i.e. hub.docker.com/r/<DOCKER_ORG>/<DOCKER_REPO>) "
-  echo "           Also used for tagging the image."
-  echo "           Default: ninech"
-  echo "  DOCKER_REPO The Docker registry (i.e. hub.docker.com/r/<DOCKER_ORG>/<DOCKER_REPO>) "
-  echo "           Also used for tagging the image."
-  echo "           Default: netbox"
-  echo "  DOCKER_TAG The name of the tag which is applied to the image."
-  echo "           Useful for pushing into another registry than hub.docker.com."
-  echo "           Default: <DOCKER_ORG>/<DOCKER_REPO>:<BRANCH>"
-  echo "  SRC_ORG  Which fork of netbox to use (i.e. github.com/<SRC_ORG>/<SRC_REPO>)."
-  echo "           Default: digitalocean"
-  echo "  SRC_REPO The name of the netbox for to use (i.e. github.com/<SRC_ORG>/<SRC_REPO>)."
-  echo "           Default: netbox"
-  echo "  URL      Where to fetch the package from."
-  echo "           Must be a tar.gz file of the source code."
-  echo "           Default: https://github.com/<SRC_ORG>/<SRC_REPO>/archive/\$BRANCH.tar.gz"
-  echo "  VARIANT  The variant to build."
-  echo "           The value will be used as a suffix to the \$TAG and for the Dockerfile"
-  echo "           selection. The TAG being build must exist for the base variant and"
-  echo "           corresponding Dockerfile must start with the following lines:"
-  echo "             ARG DOCKER_ORG=ninech"
-  echo "             ARG DOCKER_REPO=netbox"
-  echo "             ARG FROM_TAG=latest"
-  echo "             FROM \$DOCKER_ORG/\$DOCKER_REPO:\$FROM_TAG"
-  echo "           Example: VARIANT=ldap will result in the tag 'latest-ldap' and the"
-  echo "            Dockerfile 'Dockerfile.ldap' being used."
-  echo "           Default: empty"
-  echo "  HTTP_PROXY The proxy to use for http requests."
-  echo "           Example: http://proxy.domain.tld:3128"
-  echo "           Default: empty"
-  echo "  HTTPS_PROXY The proxy to use for https requests."
-  echo "           Example: http://proxy.domain.tld:3128"
-  echo "           Default: empty"
-  echo "  FTP_PROXY The proxy to use for ftp requests."
-  echo "           Example: http://proxy.domain.tld:3128"
-  echo "           Default: empty"
-  echo "  NO_PROXY Comma-separated list of domain extensions proxy should not be used for."
-  echo "           Example: .domain1.tld,.domain2.tld"
-  echo "           Default: empty"
+  echo "  SRC_ORG     Which fork of netbox to use (i.e. github.com/\${SRC_ORG}/\${SRC_REPO})."
+  echo "              Default: netbox-community"
+  echo "  SRC_REPO    The name of the repository to use (i.e. github.com/\${SRC_ORG}/\${SRC_REPO})."
+  echo "              Default: netbox"
+  echo "  URL         Where to fetch the code from."
+  echo "              Must be a git repository. Can be private."
+  echo "              Default: https://github.com/\${SRC_ORG}/\${SRC_REPO}.git"
+  echo "  NETBOX_PATH The path where netbox will be checkout out."
+  echo "              Must not be outside of the netbox-docker repository (because of Docker)!"
+  echo "              Default: .netbox"
+  echo "  SKIP_GIT    If defined, git is not invoked and \${NETBOX_PATH} will not be altered."
+  echo "              This may be useful, if you are manually managing the NETBOX_PATH."
+  echo "              Default: undefined"
+  echo "  TAG         The version part of the docker tag."
+  echo "              Default:"
+  echo "                When <branch>=master:  latest"
+  echo "                When <branch>=develop: snapshot"
+  echo "                Else:                  same as <branch>"
+  echo "  DOCKER_REGISTRY The Docker repository's registry (i.e. '\${DOCKER_REGISTRY}/\${DOCKER_ORG}/\${DOCKER_REPO}'')"
+  echo "              Used for tagging the image."
+  echo "              Default: docker.io"
+  echo "  DOCKER_ORG  The Docker repository's organisation (i.e. '\${DOCKER_REGISTRY}/\${DOCKER_ORG}/\${DOCKER_REPO}'')"
+  echo "              Used for tagging the image."
+  echo "              Default: netboxcommunity"
+  echo "  DOCKER_REPO The Docker repository's name (i.e. '\${DOCKER_REGISTRY}/\${DOCKER_ORG}/\${DOCKER_REPO}'')"
+  echo "              Used for tagging the image."
+  echo "              Default: netbox"
+  echo "  DOCKER_TAG  The name of the tag which is applied to the image."
+  echo "              Useful for pushing into another registry than hub.docker.com."
+  echo "              Default: \${DOCKER_REGISTRY}/\${DOCKER_ORG}/\${DOCKER_REPO}:\${TAG}"
+  echo "  DOCKER_SHORT_TAG The name of the short tag which is applied to the"
+  echo "              image. This is used to tag all patch releases to their"
+  echo "              containing version e.g. v2.5.1 -> v2.5"
+  echo "              Default: \${DOCKER_REGISTRY}/\${DOCKER_ORG}/\${DOCKER_REPO}:<MAJOR>.<MINOR>"
+  echo "  DOCKERFILE  The name of Dockerfile to use."
+  echo "              Default: Dockerfile"
+  echo "  DOCKER_FROM The base image to use."
+  echo "              Default: Whatever is defined as default in the Dockerfile."
+  echo "  DOCKER_TARGET A specific target to build."
+  echo "              It's currently not possible to pass multiple targets."
+  echo "              Default: main ldap"
+  echo "  HTTP_PROXY  The proxy to use for http requests."
+  echo "              Example: http://proxy.domain.tld:3128"
+  echo "              Default: undefined"
+  echo "  NO_PROXY    Comma-separated list of domain extensions proxy should not be used for."
+  echo "              Example: .domain1.tld,.domain2.tld"
+  echo "              Default: undefined"
+  echo "  DEBUG       If defined, the script does not stop when certain checks are unsatisfied."
+  echo "              Default: undefined"
+  echo "  DRY_RUN     Prints all build statements instead of running them."
+  echo "              Default: undefined"
+  echo "  GH_ACTION   If defined, special 'echo' statements are enabled that set the"
+  echo "              following environment variables in Github Actions:"
+  echo "              - FINAL_DOCKER_TAG: The final value of the DOCKER_TAG env variable"
+  echo "              Default: undefined"
+  echo ""
+  echo "Examples:"
+  echo "  ${0} master"
+  echo "              This will fetch the latest 'master' branch, build a Docker Image and tag it"
+  echo "              'netboxcommunity/netbox:latest'."
+  echo "  ${0} develop"
+  echo "              This will fetch the latest 'develop' branch, build a Docker Image and tag it"
+  echo "              'netboxcommunity/netbox:snapshot'."
+  echo "  ${0} v2.6.6"
+  echo "              This will fetch the 'v2.6.6' tag, build a Docker Image and tag it"
+  echo "              'netboxcommunity/netbox:v2.6.6' and 'netboxcommunity/netbox:v2.6'."
+  echo "  ${0} develop-2.7"
+  echo "              This will fetch the 'develop-2.7' branch, build a Docker Image and tag it"
+  echo "              'netboxcommunity/netbox:develop-2.7'."
+  echo "  SRC_ORG=cimnine ${0} feature-x"
+  echo "              This will fetch the 'feature-x' branch from https://github.com/cimnine/netbox.git,"
+  echo "              build a Docker Image and tag it 'netboxcommunity/netbox:feature-x'."
+  echo "  SRC_ORG=cimnine DOCKER_ORG=cimnine ${0} feature-x"
+  echo "              This will fetch the 'feature-x' branch from https://github.com/cimnine/netbox.git,"
+  echo "              build a Docker Image and tag it 'cimnine/netbox:feature-x'."
 
   if [ "${1}x" == "x" ]; then
     exit 1
@@ -71,93 +95,217 @@ if [ "${1}x" == "x" ] || [ "${1}" == "--help" ] || [ "${1}" == "-h" ]; then
   fi
 fi
 
-# read the project version and trim it
-# see https://stackoverflow.com/a/3232433/172132
-NETBOX_DOCKER_PROJECT_VERSION="${NETBOX_DOCKER_PROJECT_VERSION-$(sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' VERSION)}"
+###
+# Enabling dry-run mode
+###
+if [ -z "${DRY_RUN}" ]; then
+  DRY=""
+else
+  echo "⚠️ DRY_RUN MODE ON ⚠️"
+  DRY="echo"
+fi
 
-# variables for fetching the source
-SRC_ORG="${SRC_ORG-digitalocean}"
+###
+# Variables for fetching the source
+###
+SRC_ORG="${SRC_ORG-netbox-community}"
 SRC_REPO="${SRC_REPO-netbox}"
-BRANCH="${1}"
-URL="${URL-https://github.com/${SRC_ORG}/${SRC_REPO}/archive/$BRANCH.tar.gz}"
+NETBOX_BRANCH="${1}"
+URL="${URL-https://github.com/${SRC_ORG}/${SRC_REPO}.git}"
+NETBOX_PATH="${NETBOX_PATH-.netbox}"
 
-# variables for tagging the docker image
-DOCKER_ORG="${DOCKER_ORG-ninech}"
+###
+# Fetching the source
+###
+if [ "${2}" != "--push-only" ] && [ -z "${SKIP_GIT}" ] ; then
+  echo "🌐 Checking out '${NETBOX_BRANCH}' of netbox from the url '${URL}' into '${NETBOX_PATH}'"
+  if [ ! -d "${NETBOX_PATH}" ]; then
+    $DRY git clone -q --depth 10 -b "${NETBOX_BRANCH}" "${URL}" "${NETBOX_PATH}"
+  fi
+
+  (
+    $DRY cd "${NETBOX_PATH}"
+
+    if [ -n "${HTTP_PROXY}" ]; then
+      git config http.proxy "${HTTP_PROXY}"
+    fi
+
+    $DRY git remote set-url origin "${URL}"
+    $DRY git fetch -qp --depth 10 origin "${NETBOX_BRANCH}"
+    $DRY git checkout -qf FETCH_HEAD
+    $DRY git prune
+  )
+  echo "✅ Checked out netbox"
+fi
+
+###
+# Determining the value for DOCKERFILE
+# and checking whether it exists
+###
+DOCKERFILE="${DOCKERFILE-Dockerfile}"
+if [ ! -f "${DOCKERFILE}" ]; then
+  echo "🚨 The Dockerfile ${DOCKERFILE} doesn't exist."
+
+  if [ -z "${DEBUG}" ]; then
+    exit 1
+  else
+    echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
+  fi
+fi
+
+###
+# Variables for labelling the docker image
+###
+BUILD_DATE="$(date -u '+%Y-%m-%dT%H:%M+00:00')"
+
+if [ -d ".git" ]; then
+  GIT_REF="$(git rev-parse HEAD)"
+fi
+
+# Read the project version from the `VERSION` file and trim it, see https://stackoverflow.com/a/3232433/172132
+PROJECT_VERSION="${PROJECT_VERSION-$(sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' VERSION)}"
+
+# Get the Git information from the netbox directory
+if [ -d "${NETBOX_PATH}/.git" ]; then
+  NETBOX_GIT_REF=$(cd ${NETBOX_PATH}; git rev-parse HEAD)
+  NETBOX_GIT_BRANCH=$(cd ${NETBOX_PATH}; git rev-parse --abbrev-ref HEAD)
+  NETBOX_GIT_URL=$(cd ${NETBOX_PATH}; git remote get-url origin)
+fi
+
+###
+# Variables for tagging the docker image
+###
+DOCKER_REGISTRY="${DOCKER_REGISTRY-docker.io}"
+DOCKER_ORG="${DOCKER_ORG-netboxcommunity}"
 DOCKER_REPO="${DOCKER_REPO-netbox}"
-case "${BRANCH}" in
+case "${NETBOX_BRANCH}" in
   master)
     TAG="${TAG-latest}";;
   develop)
     TAG="${TAG-snapshot}";;
   *)
-    TAG="${TAG-$BRANCH}";;
+    TAG="${TAG-$NETBOX_BRANCH}";;
 esac
-DOCKER_TAG="${DOCKER_TAG-${DOCKER_ORG}/${DOCKER_REPO}:${TAG}}"
 
-# Checking which VARIANT to build
-if [ -z "$VARIANT" ]; then
-  DOCKERFILE="Dockerfile"
-else
-  DOCKERFILE="Dockerfile.${VARIANT}"
-  DOCKER_TAG="${DOCKER_TAG}-${VARIANT}"
+###
+# Determine targets to build
+###
+DEFAULT_DOCKER_TARGETS=("main" "ldap")
+DOCKER_TARGETS=( "${DOCKER_TARGET:-"${DEFAULT_DOCKER_TARGETS[@]}"}")
+echo "🏭 Building the following targets:" "${DOCKER_TARGETS[@]}"
 
-  # Fail fast
-  if [ ! -f "${DOCKERFILE}" ]; then
-    echo "🚨 The Dockerfile ${DOCKERFILE} for variant '${VARIANT}' doesn't exist."
+###
+# Build each target
+###
+export DOCKER_BUILDKIT=${DOCKER_BUILDKIT-1}
+for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do
+  echo "🏗 Building the target '${DOCKER_TARGET}'"
 
-    if [ -z "$DEBUG" ]; then
-      exit 1
-    else
-      echo "⚠️ Would exit here with code '1', but DEBUG is enabled."
+  ###
+  # composing the final TARGET_DOCKER_TAG
+  ###
+  TARGET_DOCKER_TAG="${DOCKER_TAG-${DOCKER_REGISTRY}/${DOCKER_ORG}/${DOCKER_REPO}:${TAG}}"
+  if [ "${DOCKER_TARGET}" != "main" ]; then
+    TARGET_DOCKER_TAG="${TARGET_DOCKER_TAG}-${DOCKER_TARGET}"
+  fi
+  if [ -n "${GH_ACTION}" ]; then
+    echo "::set-env name=FINAL_DOCKER_TAG::${TARGET_DOCKER_TAG}"
+    echo "::set-output name=skipped::false"
+  fi
+
+  ###
+  # composing the additional DOCKER_SHORT_TAG,
+  # i.e. "v2.6.1" becomes "v2.6",
+  # which is only relevant for version tags
+  ###
+  if [[ "${TAG}" =~ ^v([0-9]+)\.([0-9]+)\.[0-9]+$ ]]; then
+    MAJOR=${BASH_REMATCH[1]}
+    MINOR=${BASH_REMATCH[2]}
+
+    TARGET_DOCKER_SHORT_TAG="${DOCKER_SHORT_TAG-${DOCKER_REGISTRY}/${DOCKER_ORG}/${DOCKER_REPO}:v${MAJOR}.${MINOR}}"
+
+    if [ "${DOCKER_TARGET}" != "main" ]; then
+      TARGET_DOCKER_SHORT_TAG="${TARGET_DOCKER_SHORT_TAG}-${DOCKER_TARGET}"
     fi
   fi
-fi
 
-DOCKER_OPTS=("${DOCKER_OPTS[@]}")
+  ###
+  # Proceeding to buils stage, except if `--push-only` is passed
+  ###
+  if [ "${2}" != "--push-only" ] ; then
+    ###
+    # Composing all arguments for `docker build`
+    ###
+    DOCKER_BUILD_ARGS=(
+      --pull
+      --target "${DOCKER_TARGET}"
+      -f "${DOCKERFILE}"
+      -t "${TARGET_DOCKER_TAG}"
+    )
+    if [ -n "${TARGET_DOCKER_SHORT_TAG}" ]; then
+      DOCKER_BUILD_ARGS+=( -t "${TARGET_DOCKER_SHORT_TAG}" )
+    fi
 
-# caching is only ok for version tags
-case "${TAG}" in
-  v*) ;;
-  *)  DOCKER_OPTS+=( "--no-cache" ) ;;
-esac
+    # --label
+    if [ "${DOCKER_TARGET}" == "main" ]; then
+      DOCKER_BUILD_ARGS+=(
+        --label "ORIGINAL_TAG=${TARGET_DOCKER_TAG}"
 
-DOCKER_OPTS+=( "--pull" )
+        --label "org.label-schema.build-date=${BUILD_DATE}"
+        --label "org.opencontainers.image.created=${BUILD_DATE}"
 
-# Build args
-DOCKER_BUILD_ARGS=(
-  --build-arg "NETBOX_DOCKER_PROJECT_VERSION=${NETBOX_DOCKER_PROJECT_VERSION}"
-  --build-arg "FROM_TAG=${TAG}"
-  --build-arg "BRANCH=${BRANCH}"
-  --build-arg "URL=${URL}"
-  --build-arg "DOCKER_ORG=${DOCKER_ORG}"
-  --build-arg "DOCKER_REPO=${DOCKER_REPO}"
-)
-if [ -n "$HTTP_PROXY" ]; then
-  DOCKER_BUILD_ARGS+=( --build-arg "http_proxy=${HTTP_PROXY}" )
-fi
-if [ -n "$HTTPS_PROXY" ]; then
-  DOCKER_BUILD_ARGS+=( --build-arg "https_proxy=${HTTPS_PROXY}" )
-fi
-if [ -n "$FTP_PROXY" ]; then
-  DOCKER_BUILD_ARGS+=( --build-arg "ftp_proxy=${FTP_PROXY}" )
-fi
-if [ -n "$NO_PROXY" ]; then
-  DOCKER_BUILD_ARGS+=( --build-arg "no_proxy=${NO_PROXY}" )
-fi
+        --label "org.label-schema.version=${PROJECT_VERSION}"
+        --label "org.opencontainers.image.version=${PROJECT_VERSION}"
+      )
+      if [ -d ".git" ]; then
+        DOCKER_BUILD_ARGS+=(
+          --label "org.label-schema.vcs-ref=${GIT_REF}"
+          --label "org.opencontainers.image.revision=${GIT_REF}"
+        )
+      fi
+      if [ -d "${NETBOX_PATH}/.git" ]; then
+        DOCKER_BUILD_ARGS+=(
+          --label "NETBOX_GIT_BRANCH=${NETBOX_GIT_BRANCH}"
+          --label "NETBOX_GIT_REF=${NETBOX_GIT_REF}"
+          --label "NETBOX_GIT_URL=${NETBOX_GIT_URL}"
+        )
+      fi
+    fi
 
-if [ -z "$DRY_RUN" ]; then
-  DOCKER_CMD="docker"
-else
-  echo "⚠️ DRY_RUN MODE ON ⚠️"
-  DOCKER_CMD="echo docker"
-fi
+    # --build-arg
+    DOCKER_BUILD_ARGS+=(   --build-arg "NETBOX_PATH=${NETBOX_PATH}" )
 
-echo "🐳 Building the Docker image '${DOCKER_TAG}' from the url '${URL}'."
-$DOCKER_CMD build -t "${DOCKER_TAG}" "${DOCKER_BUILD_ARGS[@]}" "${DOCKER_OPTS[@]}" -f "${DOCKERFILE}" .
-echo "✅ Finished building the Docker images '${DOCKER_TAG}'"
+    if [ -n "${DOCKER_FROM}" ]; then
+      DOCKER_BUILD_ARGS+=( --build-arg "FROM=${DOCKER_FROM}" )
+    fi
+    if [ -n "${HTTP_PROXY}" ]; then
+      DOCKER_BUILD_ARGS+=( --build-arg "http_proxy=${HTTP_PROXY}" )
+      DOCKER_BUILD_ARGS+=( --build-arg "https_proxy=${HTTPS_PROXY}" )
+    fi
+    if [ -n "${NO_PROXY}" ]; then
+      DOCKER_BUILD_ARGS+=( --build-arg "no_proxy=${NO_PROXY}" )
+    fi
 
-if [ "${2}" == "--push" ] ; then
-  echo "⏫ Pushing '${DOCKER_TAG}"
-  $DOCKER_CMD push "${DOCKER_TAG}"
-  echo "✅ Finished pushing the Docker image '${DOCKER_TAG}'."
-fi
+    ###
+    # Building the docker image
+    ###
+    echo "🐳 Building the Docker image '${TARGET_DOCKER_TAG}'."
+    $DRY docker build "${DOCKER_BUILD_ARGS[@]}" .
+    echo "✅ Finished building the Docker images '${TARGET_DOCKER_TAG}'"
+  fi
+
+  ###
+  # Pushing the docker images if either `--push` or `--push-only` are passed
+  ###
+  if [ "${2}" == "--push" ] || [ "${2}" == "--push-only" ] ; then
+    echo "⏫ Pushing '${TARGET_DOCKER_TAG}"
+    $DRY docker push "${TARGET_DOCKER_TAG}"
+    echo "✅ Finished pushing the Docker image '${TARGET_DOCKER_TAG}'."
+
+    if [ -n "${TARGET_DOCKER_SHORT_TAG}" ]; then
+      echo "⏫ Pushing '${TARGET_DOCKER_SHORT_TAG}'"
+      $DRY docker push "${TARGET_DOCKER_SHORT_TAG}"
+      echo "✅ Finished pushing the Docker image '${TARGET_DOCKER_SHORT_TAG}'."
+    fi
+  fi
+done

configuration/configuration.py

@@ -1,8 +1,9 @@
 import os
+import re
 import socket
 
 # For reference see http://netbox.readthedocs.io/en/latest/configuration/mandatory-settings/
-# Based on https://github.com/digitalocean/netbox/blob/develop/netbox/netbox/configuration.example.py
+# Based on https://github.com/netbox-community/netbox/blob/develop/netbox/netbox/configuration.example.py
 
 # Read secret from file
 def read_secret(secret_name):
@@ -36,6 +37,10 @@ DATABASE = {
                                                          # PostgreSQL password
     'HOST': os.environ.get('DB_HOST', 'localhost'),      # Database server
     'PORT': os.environ.get('DB_PORT', ''),               # Database port (leave blank for default)
+    'OPTIONS': {'sslmode': os.environ.get('DB_SSLMODE', 'prefer')},
+                                                         # Database connection SSLMODE
+    'CONN_MAX_AGE': int(os.environ.get('DB_CONN_MAX_AGE', '300')),
+                                                         # Database connection persistence
 }
 
 # This key is used for secure generation of random numbers and strings. It must never be exposed outside of this file.
@@ -44,6 +49,26 @@ DATABASE = {
 # https://docs.djangoproject.com/en/dev/ref/settings/#std:setting-SECRET_KEY
 SECRET_KEY = os.environ.get('SECRET_KEY', read_secret('secret_key'))
 
+# Redis database settings. The Redis database is used for caching and background processing such as webhooks
+REDIS = {
+    'webhooks': {
+        'HOST': os.environ.get('REDIS_HOST', 'localhost'),
+        'PORT': int(os.environ.get('REDIS_PORT', 6379)),
+        'PASSWORD': os.environ.get('REDIS_PASSWORD', read_secret('redis_password')),
+        'DATABASE': int(os.environ.get('REDIS_DATABASE', 0)),
+        'DEFAULT_TIMEOUT': int(os.environ.get('REDIS_TIMEOUT', 300)),
+        'SSL': os.environ.get('REDIS_SSL', 'False').lower() == 'true',
+    },
+    'caching': {
+        'HOST': os.environ.get('REDIS_CACHE_HOST', os.environ.get('REDIS_HOST', 'localhost')),
+        'PORT': int(os.environ.get('REDIS_CACHE_PORT', os.environ.get('REDIS_PORT', 6379))),
+        'PASSWORD': os.environ.get('REDIS_CACHE_PASSWORD', os.environ.get('REDIS_PASSWORD', read_secret('redis_cache_password'))),
+        'DATABASE': int(os.environ.get('REDIS_CACHE_DATABASE', 1)),
+        'DEFAULT_TIMEOUT': int(os.environ.get('REDIS_CACHE_TIMEOUT', os.environ.get('REDIS_TIMEOUT', 300))),
+        'SSL': os.environ.get('REDIS_CACHE_SSL', os.environ.get('REDIS_SSL', 'False')).lower() == 'true',
+    },
+}
+
 #########################
 #                       #
 #   Optional settings   #
@@ -68,14 +93,18 @@ BANNER_LOGIN = os.environ.get('BANNER_LOGIN', '')
 # BASE_PATH = 'netbox/'
 BASE_PATH = os.environ.get('BASE_PATH', '')
 
+# Cache timeout in seconds. Set to 0 to dissable caching. Defaults to 900 (15 minutes)
+CACHE_TIMEOUT = int(os.environ.get('CACHE_TIMEOUT', 900))
+
+# Maximum number of days to retain logged changes. Set to 0 to retain changes indefinitely. (Default: 90)
+CHANGELOG_RETENTION = int(os.environ.get('CHANGELOG_RETENTION', 90))
+
 # API Cross-Origin Resource Sharing (CORS) settings. If CORS_ORIGIN_ALLOW_ALL is set to True, all origins will be
 # allowed. Otherwise, define a list of allowed origins using either CORS_ORIGIN_WHITELIST or
 # CORS_ORIGIN_REGEX_WHITELIST. For more information, see https://github.com/ottoyiu/django-cors-headers
 CORS_ORIGIN_ALLOW_ALL = os.environ.get('CORS_ORIGIN_ALLOW_ALL', 'False').lower() == 'true'
-CORS_ORIGIN_WHITELIST = os.environ.get('CORS_ORIGIN_WHITELIST', '').split(' ')
-CORS_ORIGIN_REGEX_WHITELIST = [
-    # r'^(https?://)?(\w+\.)?example\.com$',
-]
+CORS_ORIGIN_WHITELIST = list(filter(None, os.environ.get('CORS_ORIGIN_WHITELIST', 'https://localhost').split(' ')))
+CORS_ORIGIN_REGEX_WHITELIST = [re.compile(r) for r in list(filter(None, os.environ.get('CORS_ORIGIN_REGEX_WHITELIST', '').split(' ')))]
 
 # Set to True to enable server debugging. WARNING: Debugging introduces a substantial performance penalty and may reveal
 # sensitive information about your installation. Only enable debugging while performing testing. Never enable debugging
@@ -97,6 +126,10 @@ EMAIL = {
 # set ENFORCE_GLOBAL_UNIQUE to True.
 ENFORCE_GLOBAL_UNIQUE = os.environ.get('ENFORCE_GLOBAL_UNIQUE', 'False').lower() == 'true'
 
+# Exempt certain models from the enforcement of view permissions. Models listed here will be viewable by all users and
+# by anonymous users. List models in the form `<app>.<model>`. Add '*' to this list to exempt all models.
+EXEMPT_VIEW_PERMISSIONS = list(filter(None, os.environ.get('EXEMPT_VIEW_PERMISSIONS', '').split(' ')))
+
 # Enable custom logging. Please see the Django documentation for detailed guidance on configuring custom logs:
 #   https://docs.djangoproject.com/en/1.11/topics/logging/
 LOGGING = {}
@@ -105,10 +138,6 @@ LOGGING = {}
 # are permitted to access most data in NetBox (excluding secrets) but not make any changes.
 LOGIN_REQUIRED = os.environ.get('LOGIN_REQUIRED', 'False').lower() == 'true'
 
-# Base URL path if accessing NetBox within a directory. For example, if installed at http://example.com/netbox/, set:
-# BASE_PATH = 'netbox/'
-BASE_PATH = os.environ.get('BASE_PATH', '')
-
 # Setting this to True will display a "maintenance mode" banner at the top of every page.
 MAINTENANCE_MODE = os.environ.get('MAINTENANCE_MODE', 'False').lower() == 'true'
 
@@ -121,6 +150,9 @@ MAX_PAGE_SIZE = int(os.environ.get('MAX_PAGE_SIZE', 1000))
 # the default value of this setting is derived from the installed location.
 MEDIA_ROOT = os.environ.get('MEDIA_ROOT', os.path.join(BASE_DIR, 'media'))
 
+# Expose Prometheus monitoring metrics at the HTTP endpoint '/metrics'
+METRICS_ENABLED = os.environ.get('METRICS_ENABLED', 'False').lower() == 'true'
+
 # Credentials that NetBox will use to access live devices.
 NAPALM_USERNAME = os.environ.get('NAPALM_USERNAME', '')
 NAPALM_PASSWORD = os.environ.get('NAPALM_PASSWORD', read_secret('napalm_password'))
@@ -138,24 +170,14 @@ PAGINATE_COUNT = int(os.environ.get('PAGINATE_COUNT', 50))
 # When determining the primary IP address for a device, IPv6 is preferred over IPv4 by default. Set this to True to
 # prefer IPv4 instead.
 PREFER_IPV4 = os.environ.get('PREFER_IPV4', 'False').lower() == 'true'
-
-# The Webhook event backend is disabled by default. Set this to True to enable it. Note that this requires a Redis
-# database be configured and accessible by NetBox (see `REDIS` below).
-WEBHOOKS_ENABLED = os.environ.get('WEBHOOKS_ENABLED', 'False').lower() == 'true'
-
-# Redis database settings (optional). A Redis database is required only if the webhooks backend is enabled.
-REDIS = {
-    'HOST': os.environ.get('REDIS_HOST', 'localhost'),
-    'PORT': os.environ.get('REDIS_PORT', '6379'),
-    'PASSWORD': os.environ.get('REDIS_PASSWORD', read_secret('redis_password')),
-    'DATABASE': os.environ.get('REDIS_DATABASE', '0'),
-    'DEFAULT_TIMEOUT': os.environ.get('REDIS_TIMEOUT', '300'),
-}
-
 # The file path where custom reports will be stored. A trailing slash is not needed. Note that the default value of
 # this setting is derived from the installed location.
 REPORTS_ROOT = os.environ.get('REPORTS_ROOT', '/etc/netbox/reports')
 
+# The file path where custom scripts will be stored. A trailing slash is not needed. Note that the default value of
+# this setting is derived from the installed location.
+SCRIPTS_ROOT = os.environ.get('SCRIPTS_ROOT', '/etc/netbox/scripts')
+
 # Time zone (default: UTC)
 TIME_ZONE = os.environ.get('TIME_ZONE', 'UTC')
 

configuration/ldap_config.py

@@ -1,7 +1,26 @@
 import ldap
 import os
 
-from django_auth_ldap.config import LDAPSearch, GroupOfNamesType
+from django_auth_ldap.config import LDAPSearch
+from importlib import import_module
+
+# Read secret from file
+def read_secret(secret_name):
+    try:
+        f = open('/run/secrets/' + secret_name, 'r', encoding='utf-8')
+    except EnvironmentError:
+        return ''
+    else:
+        with f:
+            return f.readline().strip()
+
+# Import and return the group type based on string name
+def import_group_type(group_type_name):
+    mod = import_module('django_auth_ldap.config')
+    try:
+        return getattr(mod, group_type_name)()
+    except:
+        return None
 
 # Server URI
 AUTH_LDAP_SERVER_URI = os.environ.get('AUTH_LDAP_SERVER_URI', '')
@@ -13,7 +32,7 @@ AUTH_LDAP_CONNECTION_OPTIONS = {
 
 # Set the DN and password for the NetBox service account.
 AUTH_LDAP_BIND_DN = os.environ.get('AUTH_LDAP_BIND_DN', '')
-AUTH_LDAP_BIND_PASSWORD = os.environ.get('AUTH_LDAP_BIND_PASSWORD', '')
+AUTH_LDAP_BIND_PASSWORD = os.environ.get('AUTH_LDAP_BIND_PASSWORD', read_secret('auth_ldap_bind_password'))
 
 # Set a string template that describes any user’s distinguished name based on the username.
 AUTH_LDAP_USER_DN_TEMPLATE = os.environ.get('AUTH_LDAP_USER_DN_TEMPLATE', None)
@@ -23,15 +42,19 @@ AUTH_LDAP_USER_DN_TEMPLATE = os.environ.get('AUTH_LDAP_USER_DN_TEMPLATE', None)
 #     ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
 LDAP_IGNORE_CERT_ERRORS = os.environ.get('LDAP_IGNORE_CERT_ERRORS', 'False').lower() == 'true'
 
-AUTH_LDAP_USER_SEARCH = LDAPSearch(os.environ.get('AUTH_LDAP_USER_SEARCH_BASEDN', ''),
-                                    ldap.SCOPE_SUBTREE,
-                                    "(sAMAccountName=%(user)s)")
+AUTH_LDAP_USER_SEARCH_BASEDN = os.environ.get('AUTH_LDAP_USER_SEARCH_BASEDN', '')
+AUTH_LDAP_USER_SEARCH_ATTR = os.environ.get('AUTH_LDAP_USER_SEARCH_ATTR', 'sAMAccountName')
+AUTH_LDAP_USER_SEARCH = LDAPSearch(AUTH_LDAP_USER_SEARCH_BASEDN,
+                                   ldap.SCOPE_SUBTREE,
+                                   "(" + AUTH_LDAP_USER_SEARCH_ATTR + "=%(user)s)")
 
 # This search ought to return all groups to which the user belongs. django_auth_ldap uses this to determine group
 # heirarchy.
-AUTH_LDAP_GROUP_SEARCH = LDAPSearch(os.environ.get('AUTH_LDAP_GROUP_SEARCH_BASEDN', ''), ldap.SCOPE_SUBTREE,
-                                    "(objectClass=group)")
-AUTH_LDAP_GROUP_TYPE = GroupOfNamesType()
+AUTH_LDAP_GROUP_SEARCH_BASEDN = os.environ.get('AUTH_LDAP_GROUP_SEARCH_BASEDN', '')
+AUTH_LDAP_GROUP_SEARCH_CLASS = os.environ.get('AUTH_LDAP_GROUP_SEARCH_CLASS', 'group')
+AUTH_LDAP_GROUP_SEARCH = LDAPSearch(AUTH_LDAP_GROUP_SEARCH_BASEDN, ldap.SCOPE_SUBTREE,
+                                    "(objectClass=" + AUTH_LDAP_GROUP_SEARCH_CLASS + ")")
+AUTH_LDAP_GROUP_TYPE = import_group_type(os.environ.get('AUTH_LDAP_GROUP_TYPE', 'GroupOfNamesType'))
 
 # Define a group required to login.
 AUTH_LDAP_REQUIRE_GROUP = os.environ.get('AUTH_LDAP_REQUIRE_GROUP_DN', '')

docker-compose.test.yml

@@ -0,0 +1,45 @@
+version: '3.4'
+services:
+  netbox:
+    image: ${IMAGE-netboxcommunity/netbox:latest}
+    depends_on:
+    - postgres
+    - redis
+    env_file: env/netbox.env
+    user: '101'
+    volumes:
+    - ./startup_scripts:/opt/netbox/startup_scripts:z,ro
+    - ./${INITIALIZERS_DIR-initializers}:/opt/netbox/initializers:z,ro
+    - ./configuration:/etc/netbox/config:z,ro
+    - ./reports:/etc/netbox/reports:z,ro
+    - ./scripts:/etc/netbox/scripts:z,ro
+    - netbox-nginx-config:/etc/netbox-nginx:z
+    - netbox-static-files:/opt/netbox/netbox/static:z
+    - netbox-media-files:/opt/netbox/netbox/media:z
+  nginx:
+    command: nginx -c /etc/netbox-nginx/nginx.conf
+    image: nginx:1.17-alpine
+    depends_on:
+    - netbox
+    ports:
+    - 8080
+    volumes:
+    - netbox-static-files:/opt/netbox/netbox/static:ro
+    - netbox-nginx-config:/etc/netbox-nginx/:ro
+  postgres:
+    image: postgres:11-alpine
+    env_file: env/postgres.env
+  redis:
+    image: redis:5-alpine
+    command:
+    - sh
+    - -c # this is to evaluate the $REDIS_PASSWORD from the env
+    - redis-server --requirepass $$REDIS_PASSWORD ## $$ because of docker-compose
+    env_file: env/redis.env
+volumes:
+  netbox-static-files:
+    driver: local
+  netbox-nginx-config:
+    driver: local
+  netbox-media-files:
+    driver: local

docker-compose.yml

@@ -1,24 +1,23 @@
-version: '3'
+version: '3.4'
 services:
   netbox: &netbox
-    build:
-      context: .
-      args:
-        - BRANCH=${VERSION-master}
-    image: ninech/netbox:${VERSION-latest}
+    image: netboxcommunity/netbox:${VERSION-latest}
     depends_on:
     - postgres
     - redis
+    - redis-cache
     - netbox-worker
-    env_file: netbox.env
+    env_file: env/netbox.env
+    user: '101'
     volumes:
-    - ./startup_scripts:/opt/netbox/startup_scripts:ro
-    - ./initializers:/opt/netbox/initializers:ro
-    - ./configuration:/etc/netbox/config:ro
-    - netbox-nginx-config:/etc/netbox-nginx/
-    - netbox-static-files:/opt/netbox/netbox/static
-    - netbox-media-files:/opt/netbox/netbox/media
-    - netbox-report-files:/etc/netbox/reports:ro
+    - ./startup_scripts:/opt/netbox/startup_scripts:z,ro
+    - ./initializers:/opt/netbox/initializers:z,ro
+    - ./configuration:/etc/netbox/config:z,ro
+    - ./reports:/etc/netbox/reports:z,ro
+    - ./scripts:/etc/netbox/scripts:z,ro
+    - netbox-nginx-config:/etc/netbox-nginx:z
+    - netbox-static-files:/opt/netbox/netbox/static:z
+    - netbox-media-files:/opt/netbox/netbox/media:z
   netbox-worker:
     <<: *netbox
     depends_on:
@@ -30,7 +29,7 @@ services:
     - rqworker
   nginx:
     command: nginx -c /etc/netbox-nginx/nginx.conf
-    image: nginx:1.15-alpine
+    image: nginx:1.17-alpine
     depends_on:
     - netbox
     ports:
@@ -39,19 +38,26 @@ services:
     - netbox-static-files:/opt/netbox/netbox/static:ro
     - netbox-nginx-config:/etc/netbox-nginx/:ro
   postgres:
-    image: postgres:10.4-alpine
-    env_file: postgres.env
+    image: postgres:11-alpine
+    env_file: env/postgres.env
     volumes:
     - netbox-postgres-data:/var/lib/postgresql/data
   redis:
-    image: redis:4-alpine
+    image: redis:5-alpine
     command:
     - sh
     - -c # this is to evaluate the $REDIS_PASSWORD from the env
     - redis-server --appendonly yes --requirepass $$REDIS_PASSWORD ## $$ because of docker-compose
-    env_file: redis.env
+    env_file: env/redis.env
     volumes:
     - netbox-redis-data:/data
+  redis-cache:
+    image: redis:5-alpine
+    command:
+    - sh
+    - -c # this is to evaluate the $REDIS_PASSWORD from the env
+    - redis-server --requirepass $$REDIS_PASSWORD ## $$ because of docker-compose
+    env_file: env/redis.env
 volumes:
   netbox-static-files:
     driver: local
@@ -59,8 +65,6 @@ volumes:
     driver: local
   netbox-media-files:
     driver: local
-  netbox-report-files:
-    driver: local
   netbox-postgres-data:
     driver: local
   netbox-redis-data:

docker/docker-entrypoint.sh

@@ -1,37 +1,52 @@
 #!/bin/bash
+# Runs on every start of the Netbox Docker container
+
+# Stop when an error occures
 set -e
 
-# wait shortly and then run db migrations (retry on error)
-while ! ./manage.py migrate 2>&1; do
-  echo "⏳ Waiting on DB..."
-  sleep 3
+# Allows Netbox to be run as non-root users
+umask 002
+
+# Try to connect to the DB
+DB_WAIT_TIMEOUT=${DB_WAIT_TIMEOUT-3}
+MAX_DB_WAIT_TIME=${MAX_DB_WAIT_TIME-30}
+CUR_DB_WAIT_TIME=0
+while ! ./manage.py migrate 2>&1 && [ "${CUR_DB_WAIT_TIME}" -lt "${MAX_DB_WAIT_TIME}" ]; do
+  echo "⏳ Waiting on DB... (${CUR_DB_WAIT_TIME}s / ${MAX_DB_WAIT_TIME}s)"
+  sleep "${DB_WAIT_TIMEOUT}"
+  CUR_DB_WAIT_TIME=$(( CUR_DB_WAIT_TIME + DB_WAIT_TIMEOUT ))
 done
+if [ "${CUR_DB_WAIT_TIME}" -ge "${MAX_DB_WAIT_TIME}" ]; then
+  echo "❌ Waited ${MAX_DB_WAIT_TIME}s or more for the DB to become ready."
+  exit 1
+fi
 
-# create superuser silently
-if [ -z ${SUPERUSER_NAME+x} ]; then
-  SUPERUSER_NAME='admin'
-fi
-if [ -z ${SUPERUSER_EMAIL+x} ]; then
-  SUPERUSER_EMAIL='admin@example.com'
-fi
-if [ -z ${SUPERUSER_PASSWORD+x} ]; then
-  if [ -f "/run/secrets/superuser_password" ]; then
-    SUPERUSER_PASSWORD="$(< /run/secrets/superuser_password)"
-  else
-    SUPERUSER_PASSWORD='admin'
+# Create Superuser if required
+if [ "$SKIP_SUPERUSER" == "true" ]; then
+  echo "↩️ Skip creating the superuser"
+else
+  if [ -z ${SUPERUSER_NAME+x} ]; then
+    SUPERUSER_NAME='admin'
   fi
-fi
-if [ -z ${SUPERUSER_API_TOKEN+x} ]; then
-  if [ -f "/run/secrets/superuser_api_token" ]; then
-    SUPERUSER_API_TOKEN="$(< /run/secrets/superuser_api_token)"
-  else
-    SUPERUSER_API_TOKEN='0123456789abcdef0123456789abcdef01234567'
+  if [ -z ${SUPERUSER_EMAIL+x} ]; then
+    SUPERUSER_EMAIL='admin@example.com'
+  fi
+  if [ -z ${SUPERUSER_PASSWORD+x} ]; then
+    if [ -f "/run/secrets/superuser_password" ]; then
+      SUPERUSER_PASSWORD="$(< /run/secrets/superuser_password)"
+    else
+      SUPERUSER_PASSWORD='admin'
+    fi
+  fi
+  if [ -z ${SUPERUSER_API_TOKEN+x} ]; then
+    if [ -f "/run/secrets/superuser_api_token" ]; then
+      SUPERUSER_API_TOKEN="$(< /run/secrets/superuser_api_token)"
+    else
+      SUPERUSER_API_TOKEN='0123456789abcdef0123456789abcdef01234567'
+    fi
   fi
-fi
 
-echo "💡 Username: ${SUPERUSER_NAME}, E-Mail: ${SUPERUSER_EMAIL}"
-
-./manage.py shell --interface python << END
+  ./manage.py shell --interface python << END
 from django.contrib.auth.models import User
 from users.models import Token
 if not User.objects.filter(username='${SUPERUSER_NAME}'):
@@ -39,16 +54,23 @@ if not User.objects.filter(username='${SUPERUSER_NAME}'):
     Token.objects.create(user=u, key='${SUPERUSER_API_TOKEN}')
 END
 
-for script in /opt/netbox/startup_scripts/*.py; do
-  echo "⚙️ Executing '$script'"
-  ./manage.py shell --interface python < "${script}"
-done
+  echo "💡 Superuser Username: ${SUPERUSER_NAME}, E-Mail: ${SUPERUSER_EMAIL}"
+fi
 
-# copy static files
+# Run the startup scripts (and initializers)
+if [ "$SKIP_STARTUP_SCRIPTS" == "true" ]; then
+  echo "↩️ Skipping startup scripts"
+else
+  echo "import runpy; runpy.run_path('../startup_scripts')" | ./manage.py shell --interface python
+fi
+
+# Copy static files
 ./manage.py collectstatic --no-input
 
 echo "✅ Initialisation is done."
 
-# launch whatever is passed by docker
+# Launch whatever is passed by docker
 # (i.e. the RUN instruction in the Dockerfile)
-exec ${@}
+#
+# shellcheck disable=SC2068
+exec $@

env/netbox.env

@@ -1,3 +1,4 @@
+CORS_ORIGIN_ALLOW_ALL=True
 DB_NAME=netbox
 DB_USER=netbox
 DB_PASSWORD=J5brHrAXFLQSif0K
@@ -12,10 +13,18 @@ MEDIA_ROOT=/opt/netbox/netbox/media
 NAPALM_USERNAME=
 NAPALM_PASSWORD=
 NAPALM_TIMEOUT=10
-MAX_PAGE_SIZE=0
+MAX_PAGE_SIZE=1000
 REDIS_HOST=redis
 REDIS_PASSWORD=H733Kdjndks81
+REDIS_DATABASE=0
+REDIS_SSL=false
+REDIS_CACHE_HOST=redis-cache
+REDIS_CACHE_PASSWORD=t4Ph722qJ5QHeQ1qfu36
+REDIS_CACHE_DATABASE=0
+REDIS_CACHE_SSL=false
 SECRET_KEY=r8OwDznj!!dci#P9ghmRfdu1Ysxm0AiPeDCQhKE+N_rClfWNj
+SKIP_STARTUP_SCRIPTS=false
+SKIP_SUPERUSER=false
 SUPERUSER_NAME=admin
 SUPERUSER_EMAIL=admin@example.com
 SUPERUSER_PASSWORD=admin

env/redis-cache.env

@@ -0,0 +1 @@
+REDIS_PASSWORD=t4Ph722qJ5QHeQ1qfu36

initializers/aggregates.yml

@@ -0,0 +1,6 @@
+# - prefix: 10.0.0.0/16
+#   rir: RFC1918
+# - prefix: fd00:ccdd::/32
+#   rir: RFC4193 ULA
+# - prefix: 2001:db8::/32
+#   rir: RFC3849

initializers/cluster_types.yml

@@ -0,0 +1,2 @@
+# - name: Hyper-V
+#   slug: hyper-v

initializers/clusters.yml

@@ -0,0 +1,5 @@
+# - name: cluster1
+#   type: Hyper-V
+# - name: cluster2
+#   type: Hyper-V
+#   site: SING 1

initializers/custom_fields.yml

@@ -1,3 +1,18 @@
+## Possible Choices:
+##   type:
+##   - text
+##   - integer
+##   - boolean
+##   - date
+##   - url
+##   - select
+##   filter_logic:
+##   - disabled
+##   - loose
+##   - exact
+##
+## Examples:
+
 # text_field:
 #   type: text
 #   label: Custom Text
@@ -7,6 +22,8 @@
 #   on_objects:
 #   - dcim.models.Device
 #   - dcim.models.Rack
+#   - dcim.models.Site
+#   - dcim.models.DeviceType
 #   - ipam.models.IPAddress
 #   - ipam.models.Prefix
 #   - tenancy.models.Tenant
@@ -20,8 +37,8 @@
 #   weight: 10
 #   on_objects:
 #   - tenancy.models.Tenant
-# selection_field:
-#   type: selection
+# select_field:
+#   type: select
 #   label: Choose between items
 #   required: false
 #   filter_logic: exact
@@ -39,8 +56,8 @@
 #     weight: 50
 #   - value: Fourth Item
 #     weight: 40
-# selection_field_auto_weight:
-#    type: selection
+# select_field_auto_weight:
+#    type: select
 #    label: Choose between items
 #    required: false
 #    filter_logic: loose

initializers/dcim_interfaces.yml

@@ -0,0 +1,18 @@
+## Possible Choices:
+##   type:
+##   - virtual
+##   - lag
+##   - 1000base-t
+##   - ... and many more. See for yourself:
+##     https://github.com/netbox-community/netbox/blob/295d4f0394b431351c0cb2c3ecc791df68c6c2fb/netbox/dcim/choices.py#L510
+##
+## Examples:
+
+# - device: server01
+#   enabled: true
+#   type: virtual
+#   name: to-server02
+# - device: server02
+#   enabled: true
+#   type: virtual
+#   name: to-server01

initializers/device_roles.yml

@@ -0,0 +1,15 @@
+# - name: switch
+#   slug: switch
+#   color: Grey
+# - name: router
+#   slug: router
+#   color: Cyan
+# - name: load-balancer
+#   slug: load-balancer
+#   color: Red
+# - name: server
+#   slug: server
+#   color: Blue
+# - name: patchpanel
+#   slug: patchpanel
+#   color: Black

initializers/device_types.yml

@@ -0,0 +1,23 @@
+# - model: Model 1
+#   manufacturer: Manufacturer 1
+#   slug: model-1
+#   u_height: 2
+#   custom_fields:
+#     text_field: Description
+# - model: Model 2
+#   manufacturer: Manufacturer 1
+#   slug: model-2
+#   custom_fields:
+#     text_field: Description
+# - model: Model 3
+#   manufacturer: Manufacturer 1
+#   slug: model-3
+#   is_full_depth: false
+#   u_height: 0
+#   custom_fields:
+#     text_field: Description
+# - model: Other
+#   manufacturer: No Name
+#   slug: other
+#   custom_fields:
+#     text_field: Description

initializers/devices.yml

@@ -0,0 +1,42 @@
+## Possible Choices:
+##   face:
+##   - front
+##   - rear
+##   status:
+##   - offline
+##   - active
+##   - planned
+##   - staged
+##   - failed
+##   - inventory
+##   - decommissioning
+##
+## Examples:
+
+# - name: server01
+#   device_role: server
+#   device_type: Other
+#   site: AMS 1
+#   rack: rack-01
+#   face: front
+#   position: 1
+#   custom_fields:
+#     text_field: Description
+# - name: server02
+#   device_role: server
+#   device_type: Other
+#   site: AMS 2
+#   rack: rack-02
+#   face: front
+#   position: 2
+#   custom_fields:
+#     text_field: Description
+# - name: server03
+#   device_role: server
+#   device_type: Other
+#   site: SING 1
+#   rack: rack-03
+#   face: front
+#   position: 3
+#   custom_fields:
+#     text_field: Description

initializers/groups.yml

@@ -7,3 +7,10 @@
 # writers:
 #   users:
 #   - writer
+#   permissions:
+#   - add_device
+#   - change_device
+#   - delete_device
+#   - add_virtualmachine
+#   - change_virtualmachine
+#   - delete_virtualmachine

initializers/ip_addresses.yml

@@ -0,0 +1,44 @@
+## Possible Choices:
+##   status:
+##   - active
+##   - reserved
+##   - deprecated
+##   - dhcp
+##   role:
+##   - loopback
+##   - secondary
+##   - anycast
+##   - vip
+##   - vrrp
+##   - hsrp
+##   - glbp
+##   - carp
+##
+## Examples:
+
+# - address: 10.1.1.1/24
+#   device: server01
+#   interface: to-server02
+#   status: active
+#   vrf: vrf1
+# - address: 2001:db8:a000:1::1/64
+#   device: server01
+#   interface: to-server02
+#   status: active
+#   vrf: vrf1
+# - address: 10.1.1.2/24
+#   device: server02
+#   interface: to-server01
+#   status: active
+# - address: 2001:db8:a000:1::2/64
+#   device: server02
+#   interface: to-server01
+#   status: active
+# - address: 10.1.1.10/24
+#   description: reserved IP
+#   status: reserved
+#   tenant: tenant1
+# - address: 2001:db8:a000:1::10/64
+#   description: reserved IP
+#   status: reserved
+#   tenant: tenant1

initializers/manufacturers.yml

@@ -0,0 +1,6 @@
+# - name: Manufacturer 1
+#   slug: manufacturer-1
+# - name: Manufacturer 2
+#   slug: manufacturer-2
+# - name: No Name
+#   slug: no-name

initializers/platforms.yml

@@ -0,0 +1,15 @@
+# - name: Platform 1
+#   slug: platform-1
+#   manufacturer: Manufacturer 1
+#   napalm_driver: driver1
+#   napalm_args: "{'arg1': 'value1', 'arg2': 'value2'}"
+# - name: Platform 2
+#   slug: platform-2
+#   manufacturer: Manufacturer 2
+#   napalm_driver: driver2
+#   napalm_args: "{'arg1': 'value1', 'arg2': 'value2'}"
+# - name: Platform 3
+#   slug: platform-3
+#   manufacturer: No Name
+#   napalm_driver: driver3
+#   napalm_args: "{'arg1': 'value1', 'arg2': 'value2'}"

initializers/prefix_vlan_roles.yml

@@ -0,0 +1,2 @@
+# - name: Main Management
+#   slug: main-management

initializers/prefixes.yml

@@ -0,0 +1,29 @@
+## Possible Choices:
+##   status:
+##   - container
+##   - active
+##   - reserved
+##   - deprecated
+##
+## Examples:
+
+# - description: prefix1
+#   prefix: 10.1.1.0/24
+#   site: AMS 1
+#   status: active
+#   tenant: tenant1
+#   vlan: vlan1
+# - description: prefix2
+#   prefix: 10.1.2.0/24
+#   site: AMS 2
+#   status: active
+#   tenant: tenant2
+#   vlan: vlan2
+#   is_pool: true
+#   vrf: vrf2
+# - description: ipv6 prefix1
+#   prefix: 2001:db8:a000:1::/64
+#   site: AMS 2
+#   status: active
+#   tenant: tenant2
+#   vlan: vlan2

initializers/rack_groups.yml

@@ -0,0 +1,3 @@
+# - name: cage 101
+#   slug: cage-101
+#   site: SING 1

initializers/rack_roles.yml

@@ -0,0 +1,12 @@
+# - name: Role 1
+#   slug: role-1
+#   color: Pink
+# - name: Role 2
+#   slug: role-2
+#   color: Cyan
+# - name: Role 3
+#   slug: role-3
+#   color: Grey
+# - name: Role 4
+#   slug: role-4
+#   color: Teal

initializers/racks.yml

@@ -0,0 +1,41 @@
+## Possible Choices:
+##   width:
+##   - 19
+##   - 23
+##   types:
+##   - 2-post-frame
+##   - 4-post-frame
+##   - 4-post-cabinet
+##   - wall-frame
+##   - wall-cabinet
+##   outer_unit:
+##   - mm
+##   - in
+##
+## Examples:
+
+# - site: AMS 1
+#   name: rack-01
+#   role: Role 1
+#   type: 4-post-cabinet
+#   width: 19
+#   u_height: 47
+#   custom_fields:
+#     text_field: Description
+# - site: AMS 2
+#   name: rack-02
+#   role: Role 2
+#   type: 4-post-cabinet
+#   width: 19
+#   u_height: 47
+#   custom_fields:
+#     text_field: Description
+# - site: SING 1
+#   name: rack-03
+#   group: cage 101
+#   role: Role 3
+#   type: 4-post-cabinet
+#   width: 19
+#   u_height: 47
+#   custom_fields:
+#     text_field: Description

initializers/regions.yml

@@ -0,0 +1,10 @@
+# - name: Singapore
+#   slug: singapore
+# - name: Amsterdam
+#   slug: amsterdam
+# - name: Downtown
+#   slug: downtown
+#   parent: Amsterdam
+# - name: Suburbs
+#   slug: suburbs
+#   parent: Amsterdam

initializers/rirs.yml

@@ -0,0 +1,9 @@
+# - is_private: true
+#   name: RFC1918
+#   slug: rfc1918
+# - is_private: true
+#   name: RFC4193 ULA
+#   slug: rfc4193-ula
+# - is_private: true
+#   name: RFC3849
+#   slug: rfc3849

initializers/sites.yml

@@ -0,0 +1,32 @@
+# - name: AMS 1
+#   slug: ams1
+#   region: Downtown
+#   status: 1
+#   facility: Amsterdam 1
+#   asn: 12345
+#   custom_fields:
+#     text_field: Description
+# - name: AMS 2
+#   slug: ams2
+#   region: Downtown
+#   status: 1
+#   facility: Amsterdam 2
+#   asn: 54321
+#   custom_fields:
+#     text_field: Description
+# - name: AMS 3
+#   slug: ams3
+#   region: Suburbs
+#   status: 1
+#   facility: Amsterdam 3
+#   asn: 67890
+#   custom_fields:
+#     text_field: Description
+# - name: SING 1
+#   slug: sing1
+#   region: Singapore
+#   status: 1
+#   facility: Singapore 1
+#   asn: 09876
+#   custom_fields:
+#     text_field: Description

initializers/tenant_groups.yml

@@ -0,0 +1,4 @@
+# - name: Tenant Group 1
+#   slug: tenant-group-1
+# - name: Tenant Group 2
+#   slug: tenant-group-2

initializers/tenants.yml

@@ -0,0 +1,5 @@
+# - name: tenant1
+#   slug: tenant1
+# - name: tenant2
+#   slug: tenant2
+#   group: Tenant Group 2

initializers/users.yml

@@ -4,3 +4,10 @@
 #   password: reader
 # writer:
 #   password: writer
+#   permissions:
+#   - add_device
+#   - change_device
+#   - delete_device
+#   - add_virtualmachine
+#   - change_virtualmachine
+#   - delete_virtualmachine

initializers/virtual_machines.yml

@@ -0,0 +1,26 @@
+## Possible Choices:
+##   status:
+##   - active
+##   - offline
+##   - staged
+##
+## Examples:
+
+# - cluster: cluster1
+#   comments: VM1
+#   disk: 200
+#   memory: 4096
+#   name: virtual machine 1
+#   platform: Platform 2
+#   status: active
+#   tenant: tenant1
+#   vcpus: 8
+# - cluster: cluster1
+#   comments: VM2
+#   disk: 100
+#   memory: 2048
+#   name: virtual machine 2
+#   platform: Platform 2
+#   status: active
+#   tenant: tenant1
+#   vcpus: 8

initializers/virtualization_interfaces.yml

@@ -0,0 +1,12 @@
+# - description: Network Interface 1
+#   enabled: true
+#   mac_address: 00:77:77:77:77:77
+#   mtu: 1500
+#   name: Network Interface 1
+#   virtual_machine: virtual machine 1
+# - description: Network Interface 2
+#   enabled: true
+#   mac_address: 00:55:55:55:55:55
+#   mtu: 1500
+#   name: Network Interface 2
+#   virtual_machine: virtual machine 1

initializers/vlan_groups.yml

@@ -0,0 +1,6 @@
+# - name: VLAN group 1
+#   site: AMS 1
+#   slug: vlan-group-1
+# - name: VLAN group 2
+#   site: AMS 1
+#   slug: vlan-group-2

initializers/vlans.yml

@@ -0,0 +1,19 @@
+## Possible Choices:
+##   status:
+##   - active
+##   - reserved
+##   - deprecated
+##
+## Examples:
+
+# - name: vlan1
+#   site: AMS 1
+#   status: active
+#   vid: 5
+#   role: Main Management
+#   description: VLAN 5 for MGMT
+# - group: VLAN group 2
+#   name: vlan2
+#   site: AMS 1
+#   status: active
+#   vid: 1300

initializers/vrfs.yml

@@ -0,0 +1,8 @@
+# - enforce_unique: true
+#   name: vrf1
+#   tenant: tenant1
+#   description: main VRF
+# - enforce_unique: true
+#   name: vrf2
+#   rd: "6500:6500"
+#   tenant: tenant2

reports/devices.py.example

@@ -0,0 +1,46 @@
+from dcim.constants import CONNECTION_STATUS_PLANNED, DEVICE_STATUS_ACTIVE
+from dcim.models import ConsolePort, Device, PowerPort
+from extras.reports import Report
+
+
+class DeviceConnectionsReport(Report):
+    description = "Validate the minimum physical connections for each device"
+
+    def test_console_connection(self):
+
+        # Check that every console port for every active device has a connection defined.
+        for console_port in ConsolePort.objects.select_related('device').filter(device__status=DEVICE_STATUS_ACTIVE):
+            if console_port.connected_endpoint is None:
+                self.log_failure(
+                    console_port.device,
+                    "No console connection defined for {}".format(console_port.name)
+                )
+            elif console_port.connection_status == CONNECTION_STATUS_PLANNED:
+                self.log_warning(
+                    console_port.device,
+                    "Console connection for {} marked as planned".format(console_port.name)
+                )
+            else:
+                self.log_success(console_port.device)
+
+    def test_power_connections(self):
+
+        # Check that every active device has at least two connected power supplies.
+        for device in Device.objects.filter(status=DEVICE_STATUS_ACTIVE):
+            connected_ports = 0
+            for power_port in PowerPort.objects.filter(device=device):
+                if power_port.connected_endpoint is not None:
+                    connected_ports += 1
+                    if power_port.connection_status == CONNECTION_STATUS_PLANNED:
+                        self.log_warning(
+                            device,
+                            "Power connection for {} marked as planned".format(power_port.name)
+                        )
+            if connected_ports < 2:
+                self.log_failure(
+                    device,
+                    "{} connected power supplies found (2 needed)".format(connected_ports)
+                )
+            else:
+                self.log_success(device)
+

startup_scripts/000_users.py

@@ -0,0 +1,34 @@
+from django.contrib.auth.models import Permission, Group, User
+from users.models import Token
+
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/users.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml=YAML(typ='safe')
+  users = yaml.load(stream)
+
+  if users is not None:
+    for username, user_details in users.items():
+      if not User.objects.filter(username=username):
+        user = User.objects.create_user(
+          username = username,
+          password = user_details.get('password', 0) or User.objects.make_random_password)
+
+        print("👤 Created user ",username)
+
+        if user_details.get('api_token', 0):
+          Token.objects.create(user=user, key=user_details['api_token'])
+
+        user_permissions = user_details.get('permissions', [])
+        if user_permissions:
+          user.user_permissions.clear()
+          for permission_codename in user_details.get('permissions', []):
+            for permission in Permission.objects.filter(codename=permission_codename):
+              user.user_permissions.add(permission)
+          user.save()

startup_scripts/00_users.py

@@ -1,20 +0,0 @@
-from django.contrib.auth.models import Group, User
-from users.models import Token
-
-from ruamel.yaml import YAML
-
-with open('/opt/netbox/initializers/users.yml', 'r') as stream:
-  yaml=YAML(typ='safe')
-  users = yaml.load(stream)
-
-  if users is not None:
-    for username, user_details in users.items():
-      if not User.objects.filter(username=username):
-        user = User.objects.create_user(
-          username = username,
-          password = user_details.get('password', 0) or User.objects.make_random_password)
-
-        print("👤 Created user ",username)
-
-        if user_details.get('api_token', 0):
-          Token.objects.create(user=user, key=user_details['api_token'])

startup_scripts/010_groups.py

@@ -0,0 +1,32 @@
+from django.contrib.auth.models import Permission, Group, User
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/groups.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml=YAML(typ='safe')
+  groups = yaml.load(stream)
+
+  if groups is not None:
+    for groupname, group_details in groups.items():
+      group, created = Group.objects.get_or_create(name=groupname)
+
+      if created:
+        print("👥 Created group", groupname)
+
+      for username in group_details.get('users', []):
+        user = User.objects.get(username=username)
+
+        if user:
+          user.groups.add(group)
+
+      group_permissions = group_details.get('permissions', [])
+      if group_permissions:
+        group.permissions.clear()
+        for permission_codename in group_details.get('permissions', []):
+          for permission in Permission.objects.filter(codename=permission_codename):
+            group.permissions.add(permission)

startup_scripts/020_custom_fields.py

@@ -1,16 +1,8 @@
-from extras.constants import CF_TYPE_TEXT, CF_TYPE_INTEGER, CF_TYPE_BOOLEAN, CF_TYPE_DATE, CF_TYPE_URL, CF_TYPE_SELECT, CF_FILTER_CHOICES
 from extras.models import CustomField, CustomFieldChoice
 
 from ruamel.yaml import YAML
-
-text_to_fields = {
-  'boolean': CF_TYPE_BOOLEAN,
-  'date': CF_TYPE_DATE,
-  'integer': CF_TYPE_INTEGER,
-  'selection': CF_TYPE_SELECT,
-  'text': CF_TYPE_TEXT,
-  'url': CF_TYPE_URL,
-}
+from pathlib import Path
+import sys
 
 def get_class_for_class_path(class_path):
   import importlib
@@ -21,7 +13,11 @@ def get_class_for_class_path(class_path):
   clazz = getattr(module, class_name)
   return ContentType.objects.get_for_model(clazz)
 
-with open('/opt/netbox/initializers/custom_fields.yml', 'r') as stream:
+file = Path('/opt/netbox/initializers/custom_fields.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
   yaml = YAML(typ='safe')
   customfields = yaml.load(stream)
 
@@ -36,12 +32,6 @@ with open('/opt/netbox/initializers/custom_fields.yml', 'r') as stream:
         if cf_details.get('description', 0):
           custom_field.description = cf_details['description']
 
-        # If no filter_logic is specified then it will default to 'Loose'
-        if cf_details.get('filter_logic', 0):
-          for choice_id, choice_text in CF_FILTER_CHOICES:
-            if choice_text.lower() == cf_details['filter_logic']:
-              custom_field.filter_logic = choice_id
-
         if cf_details.get('label', 0):
           custom_field.label = cf_details['label']
 
@@ -52,7 +42,7 @@ with open('/opt/netbox/initializers/custom_fields.yml', 'r') as stream:
           custom_field.required = cf_details['required']
 
         if cf_details.get('type', 0):
-          custom_field.type = text_to_fields[cf_details['type']]
+          custom_field.type = cf_details['type']
 
         if cf_details.get('weight', 0):
           custom_field.weight = cf_details['weight']

startup_scripts/030_regions.py

@@ -0,0 +1,31 @@
+from dcim.models import Region
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/regions.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml=YAML(typ='safe')
+  regions = yaml.load(stream)
+
+  optional_assocs = {
+    'parent': (Region, 'name')
+  }
+
+  if regions is not None:
+    for params in regions:
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      region, created = Region.objects.get_or_create(**params)
+
+      if created:
+        print("🌐 Created region", region.name)

startup_scripts/040_sites.py

@@ -0,0 +1,46 @@
+from dcim.models import Region, Site
+from extras.models import CustomField, CustomFieldValue
+from tenancy.models import Tenant
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/sites.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  sites = yaml.load(stream)
+
+  optional_assocs = {
+    'region': (Region, 'name'),
+    'tenant': (Tenant, 'name')
+  }
+
+  if sites is not None:
+    for params in sites:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      site, created = Site.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=site,
+              value=cf_value
+            )
+
+            site.custom_field_values.add(custom_field_value)
+
+        print("📍 Created site", site.name)

startup_scripts/050_manufacturers.py

@@ -0,0 +1,19 @@
+from dcim.models import Manufacturer
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/manufacturers.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  manufacturers = yaml.load(stream)
+
+  if manufacturers is not None:
+    for params in manufacturers:
+      manufacturer, created = Manufacturer.objects.get_or_create(**params)
+
+      if created:
+        print("🏭 Created Manufacturer", manufacturer.name)

startup_scripts/060_device_types.py

@@ -0,0 +1,56 @@
+from dcim.models import DeviceType, Manufacturer, Region
+from tenancy.models import Tenant
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/device_types.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  device_types = yaml.load(stream)
+
+  required_assocs = {
+    'manufacturer': (Manufacturer, 'name')
+  }
+
+  optional_assocs = {
+    'region': (Region, 'name'),
+    'tenant': (Tenant, 'name')
+  }
+
+  if device_types is not None:
+    for params in device_types:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      device_type, created = DeviceType.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=device_type,
+              value=cf_value
+            )
+
+            device_type.custom_field_values.add(custom_field_value)
+
+        print("🔡 Created device type", device_type.manufacturer, device_type.model)

startup_scripts/070_rack_roles.py

@@ -0,0 +1,28 @@
+from dcim.models import RackRole
+from ruamel.yaml import YAML
+from utilities.forms import COLOR_CHOICES
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/rack_roles.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml=YAML(typ='safe')
+  rack_roles = yaml.load(stream)
+
+  if rack_roles is not None:
+    for params in rack_roles:
+      if 'color' in params:
+        color = params.pop('color')
+
+        for color_tpl in COLOR_CHOICES:
+          if color in color_tpl:
+            params['color'] = color_tpl[0]
+
+      rack_role, created = RackRole.objects.get_or_create(**params)
+
+      if created:
+        print("🎨 Created rack role", rack_role.name)

startup_scripts/075_rack_groups.py

@@ -0,0 +1,31 @@
+from dcim.models import Site,RackGroup
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/rack_groups.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml=YAML(typ='safe')
+  rack_groups= yaml.load(stream)
+
+  required_assocs = {
+    'site': (Site, 'name')
+  }
+
+  if rack_groups is not None:
+    for params in rack_groups:
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+        params[assoc] = model.objects.get(**query)
+
+      rack_group, created = RackGroup.objects.get_or_create(**params)
+
+      if created:
+        print("🎨 Created rack group", rack_group.name)
+

startup_scripts/080_racks.py

@@ -0,0 +1,57 @@
+from dcim.models import Site, RackRole, Rack, RackGroup
+from tenancy.models import Tenant
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/racks.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  racks = yaml.load(stream)
+
+  required_assocs = {
+    'site': (Site, 'name')
+  }
+
+  optional_assocs = {
+    'role': (RackRole, 'name'),
+    'tenant': (Tenant, 'name'),
+    'group': (RackGroup, 'name')
+  }
+
+  if racks is not None:
+    for params in racks:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      rack, created = Rack.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=rack,
+              value=cf_value
+            )
+
+            rack.custom_field_values.add(custom_field_value)
+
+        print("🔳 Created rack", rack.site, rack.name)

startup_scripts/090_device_roles.py

@@ -0,0 +1,29 @@
+from dcim.models import DeviceRole
+from ruamel.yaml import YAML
+from utilities.forms import COLOR_CHOICES
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/device_roles.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml=YAML(typ='safe')
+  device_roles = yaml.load(stream)
+
+  if device_roles is not None:
+    for params in device_roles:
+
+      if 'color' in params:
+        color = params.pop('color')
+
+        for color_tpl in COLOR_CHOICES:
+          if color in color_tpl:
+            params['color'] = color_tpl[0]
+
+      device_role, created = DeviceRole.objects.get_or_create(**params)
+
+      if created:
+        print("🎨 Created device role", device_role.name)

startup_scripts/100_platforms.py

@@ -0,0 +1,32 @@
+from dcim.models import Manufacturer, Platform
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/platforms.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  platforms = yaml.load(stream)
+
+  optional_assocs = {
+    'manufacturer': (Manufacturer, 'name'),
+  }
+
+  if platforms is not None:
+    for params in platforms:
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      platform, created = Platform.objects.get_or_create(**params)
+
+      if created:
+        print("💾 Created platform", platform.name)

startup_scripts/10_groups.py

@@ -1,19 +0,0 @@
-from django.contrib.auth.models import Group, User
-from ruamel.yaml import YAML
-
-with open('/opt/netbox/initializers/groups.yml', 'r') as stream:
-  yaml=YAML(typ='safe')
-  groups = yaml.load(stream)
-
-  if groups is not None:
-    for groupname, group_details in groups.items():
-      group, created = Group.objects.get_or_create(name=groupname)
-
-      if created:
-        print("👥 Created group", groupname)
-
-      for username in group_details['users']:
-        user = User.objects.get(username=username)
-
-        if user:
-          user.groups.add(group)

startup_scripts/110_tenant_groups.py

@@ -0,0 +1,19 @@
+from tenancy.models import TenantGroup
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/tenant_groups.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  tenant_groups = yaml.load(stream)
+
+  if tenant_groups is not None:
+    for params in tenant_groups:
+      tenant_group, created = TenantGroup.objects.get_or_create(**params)
+
+      if created:
+        print("🔳 Created Tenant Group", tenant_group.name)

startup_scripts/120_tenants.py

@@ -0,0 +1,45 @@
+from tenancy.models import Tenant, TenantGroup
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/tenants.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  tenants = yaml.load(stream)
+
+  optional_assocs = {
+    'group': (TenantGroup, 'name')
+  }
+
+  if tenants is not None:
+    for params in tenants:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      tenant, created = Tenant.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=tenant,
+              value=cf_value
+            )
+
+            tenant.custom_field_values.add(custom_field_value)
+
+        print("👩‍💻 Created Tenant", tenant.name)

startup_scripts/130_devices.py

@@ -0,0 +1,65 @@
+from dcim.models import Site, Rack, DeviceRole, DeviceType, Device, Platform
+from ipam.models import IPAddress
+from virtualization.models import Cluster
+from tenancy.models import Tenant
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/devices.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  devices = yaml.load(stream)
+
+  required_assocs = {
+    'device_role': (DeviceRole, 'name'),
+    'device_type': (DeviceType, 'model'),
+    'site': (Site, 'name')
+  }
+
+  optional_assocs = {
+    'tenant': (Tenant, 'name'),
+    'platform': (Platform, 'name'),
+    'rack': (Rack, 'name'),
+    'cluster': (Cluster, 'name'),
+    'primary_ip4': (IPAddress, 'address'),
+    'primary_ip6': (IPAddress, 'address')
+  }
+
+  if devices is not None:
+    for params in devices:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      device, created = Device.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=device,
+              value=cf_value
+            )
+
+            device.custom_field_values.add(custom_field_value)
+
+        print("🖥️  Created device", device.name)

startup_scripts/140_cluster_types.py

@@ -0,0 +1,19 @@
+from virtualization.models import ClusterType
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/cluster_types.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  cluster_types = yaml.load(stream)
+
+  if cluster_types is not None:
+    for params in cluster_types:
+      cluster_type, created = ClusterType.objects.get_or_create(**params)
+
+      if created:
+        print("🧰 Created Cluster Type", cluster_type.name)

startup_scripts/150_rirs.py

@@ -0,0 +1,19 @@
+from ipam.models import RIR
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/rirs.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  rirs = yaml.load(stream)
+
+  if rirs is not None:
+    for params in rirs:
+      rir, created = RIR.objects.get_or_create(**params)
+
+      if created:
+        print("🗺️ Created RIR", rir.name)

startup_scripts/160_aggregates.py

@@ -0,0 +1,46 @@
+from ipam.models import Aggregate, RIR
+from ruamel.yaml import YAML
+from extras.models import CustomField, CustomFieldValue
+
+from netaddr import IPNetwork
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/aggregates.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  aggregates = yaml.load(stream)
+
+  required_assocs = {
+    'rir': (RIR, 'name')
+  }
+
+  if aggregates is not None:
+    for params in aggregates:
+      custom_fields = params.pop('custom_fields', None)
+      params['prefix'] = IPNetwork(params['prefix'])
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      aggregate, created = Aggregate.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=aggregate,
+              value=cf_value
+            )
+
+            aggregate.custom_field_values.add(custom_field_value)
+
+        print("🗞️ Created Aggregate", aggregate.prefix)

startup_scripts/170_clusters.py

@@ -0,0 +1,57 @@
+from dcim.models import Site
+from virtualization.models import Cluster, ClusterType, ClusterGroup
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/clusters.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  clusters = yaml.load(stream)
+
+  required_assocs = {
+    'type': (ClusterType, 'name')
+  }
+
+  optional_assocs = {
+    'site': (Site, 'name'),
+    'group': (ClusterGroup, 'name')
+  }
+
+  if clusters is not None:
+    for params in clusters:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      cluster, created = Cluster.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=cluster,
+              value=cf_value
+            )
+
+            cluster.custom_field_values.add(custom_field_value)
+
+        print("🗄️ Created cluster", cluster.name)

startup_scripts/180_vrfs.py

@@ -0,0 +1,46 @@
+from ipam.models import VRF
+from tenancy.models import Tenant
+from ruamel.yaml import YAML
+from extras.models import CustomField, CustomFieldValue
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/vrfs.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  vrfs = yaml.load(stream)
+
+  optional_assocs = {
+    'tenant': (Tenant, 'name')
+  }
+
+  if vrfs is not None:
+    for params in vrfs:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      vrf, created = VRF.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=vrf,
+              value=cf_value
+            )
+
+            vrf.custom_field_values.add(custom_field_value)
+
+        print("📦 Created VRF", vrf.name)

startup_scripts/190_prefix_vlan_roles.py

@@ -0,0 +1,19 @@
+from ipam.models import Role
+from ruamel.yaml import YAML
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/prefix_vlan_roles.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  roles = yaml.load(stream)
+
+  if roles is not None:
+    for params in roles:
+      role, created = Role.objects.get_or_create(**params)
+
+      if created:
+        print("⛹️‍ Created Prefix/VLAN Role", role.name)

startup_scripts/200_vlan_groups.py

@@ -0,0 +1,46 @@
+from dcim.models import Site
+from ipam.models import VLANGroup
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/vlan_groups.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  vlan_groups = yaml.load(stream)
+
+  optional_assocs = {
+    'site': (Site, 'name')
+  }
+
+  if vlan_groups is not None:
+    for params in vlan_groups:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      vlan_group, created = VLANGroup.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=vlan_group,
+              value=cf_value
+            )
+
+            vlan_group.custom_field_values.add(custom_field_value)
+
+        print("🏘️ Created VLAN Group", vlan_group.name)

startup_scripts/210_vlans.py

@@ -0,0 +1,51 @@
+from dcim.models import Site
+from ipam.models import VLAN, VLANGroup, Role
+from tenancy.models import Tenant, TenantGroup
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/vlans.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  vlans = yaml.load(stream)
+
+  optional_assocs = {
+    'site': (Site, 'name'),
+    'tenant': (Tenant, 'name'),
+    'tenant_group': (TenantGroup, 'name'),
+    'group': (VLANGroup, 'name'),
+    'role': (Role, 'name')
+  }
+
+  if vlans is not None:
+    for params in vlans:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      vlan, created = VLAN.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=vlan,
+              value=cf_value
+            )
+
+            vlan.custom_field_values.add(custom_field_value)
+
+        print("🏠 Created VLAN", vlan.name)

startup_scripts/220_prefixes.py

@@ -0,0 +1,54 @@
+from dcim.models import Site
+from ipam.models import Prefix, VLAN, Role, VRF
+from tenancy.models import Tenant, TenantGroup
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from netaddr import IPNetwork
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/prefixes.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  prefixes = yaml.load(stream)
+
+  optional_assocs = {
+    'site': (Site, 'name'),
+    'tenant': (Tenant, 'name'),
+    'tenant_group': (TenantGroup, 'name'),
+    'vlan': (VLAN, 'name'),
+    'role': (Role, 'name'),
+    'vrf': (VRF, 'name')
+  }
+
+  if prefixes is not None:
+    for params in prefixes:
+      custom_fields = params.pop('custom_fields', None)
+      params['prefix'] = IPNetwork(params['prefix'])
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      prefix, created = Prefix.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=prefix,
+              value=cf_value
+            )
+
+            prefix.custom_field_values.add(custom_field_value)
+
+        print("📌 Created Prefix", prefix.prefix)

startup_scripts/230_virtual_machines.py

@@ -0,0 +1,59 @@
+from dcim.models import Site, Platform, DeviceRole
+from virtualization.models import Cluster, VirtualMachine
+from tenancy.models import Tenant
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/virtual_machines.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  virtual_machines = yaml.load(stream)
+
+  required_assocs = {
+    'cluster': (Cluster, 'name')
+  }
+
+  optional_assocs = {
+    'tenant': (Tenant, 'name'),
+    'platform': (Platform, 'name'),
+    'role': (DeviceRole, 'name')
+  }
+
+  if virtual_machines is not None:
+    for params in virtual_machines:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          query = { field: params.pop(assoc) }
+
+          params[assoc] = model.objects.get(**query)
+
+      virtual_machine, created = VirtualMachine.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=virtual_machine,
+              value=cf_value
+            )
+
+            virtual_machine.custom_field_values.add(custom_field_value)
+
+        print("🖥️ Created virtual machine", virtual_machine.name)

startup_scripts/240_virtualization_interfaces.py

@@ -0,0 +1,45 @@
+from dcim.models import Interface
+from virtualization.models import VirtualMachine
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/virtualization_interfaces.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  interfaces = yaml.load(stream)
+
+  required_assocs = {
+    'virtual_machine': (VirtualMachine, 'name')
+  }
+
+  if interfaces is not None:
+    for params in interfaces:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      interface, created = Interface.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=interface,
+              value=cf_value
+            )
+
+            interface.custom_field_values.add(custom_field_value)
+
+        print("🧷 Created interface", interface.name, interface.virtual_machine.name)

startup_scripts/250_dcim_interfaces.py

@@ -0,0 +1,44 @@
+from dcim.models import Interface, Device
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/dcim_interfaces.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  interfaces = yaml.load(stream)
+
+  required_assocs = {
+    'device': (Device, 'name')
+  }
+
+  if interfaces is not None:
+    for params in interfaces:
+      custom_fields = params.pop('custom_fields', None)
+
+      for assoc, details in required_assocs.items():
+        model, field = details
+        query = { field: params.pop(assoc) }
+
+        params[assoc] = model.objects.get(**query)
+
+      interface, created = Interface.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=interface,
+              value=cf_value
+            )
+
+            interface.custom_field_values.add(custom_field_value)
+
+        print("🧷 Created interface", interface.name, interface.device.name)

startup_scripts/260_ip_addresses.py

@@ -0,0 +1,65 @@
+from ipam.models import IPAddress, VRF
+from dcim.models import Device, Interface
+from virtualization.models import VirtualMachine
+from tenancy.models import Tenant
+from extras.models import CustomField, CustomFieldValue
+from ruamel.yaml import YAML
+
+from netaddr import IPNetwork
+from pathlib import Path
+import sys
+
+file = Path('/opt/netbox/initializers/ip_addresses.yml')
+if not file.is_file():
+  sys.exit()
+
+with file.open('r') as stream:
+  yaml = YAML(typ='safe')
+  ip_addresses = yaml.load(stream)
+
+  optional_assocs = {
+    'tenant': (Tenant, 'name'),
+    'vrf': (VRF, 'name'),
+    'interface': (Interface, 'name')
+  }
+
+  if ip_addresses is not None:
+    for params in ip_addresses:
+      vm = params.pop('virtual_machine', None)
+      device = params.pop('device', None)
+      custom_fields = params.pop('custom_fields', None)
+      params['address'] = IPNetwork(params['address'])
+
+      if vm and device:
+        print("IP Address can only specify one of the following: virtual_machine or device.")
+        sys.exit()
+
+      for assoc, details in optional_assocs.items():
+        if assoc in params:
+          model, field = details
+          if assoc == 'interface':
+              if vm:
+                  vm_id = VirtualMachine.objects.get(name=vm).id
+                  query = { field: params.pop(assoc), "virtual_machine_id": vm_id }
+              elif device:
+                  dev_id = Device.objects.get(name=device).id
+                  query = { field: params.pop(assoc), "device_id": dev_id }
+          else:
+              query = { field: params.pop(assoc) }
+          params[assoc] = model.objects.get(**query)
+
+      ip_address, created = IPAddress.objects.get_or_create(**params)
+
+      if created:
+        if custom_fields is not None:
+          for cf_name, cf_value in custom_fields.items():
+            custom_field = CustomField.objects.get(name=cf_name)
+            custom_field_value = CustomFieldValue.objects.create(
+              field=custom_field,
+              obj=ip_address,
+              value=cf_value
+            )
+
+            ip_address.custom_field_values.add(custom_field_value)
+
+        print("🧬 Created IP Address", ip_address.address)

startup_scripts/__main__.py

@@ -0,0 +1,18 @@
+#!/usr/bin/env python3
+
+import runpy
+from os import scandir
+from os.path import dirname, abspath
+
+this_dir = dirname(abspath(__file__))
+
+def filename(f):
+    return f.name
+
+with scandir(dirname(abspath(__file__))) as it:
+    for f in sorted(it, key = filename):
+        if f.name.startswith('__') or not f.is_file():
+            continue
+        
+        print(f"Running {f.path}")
+        runpy.run_path(f.path)

test.sh

@@ -0,0 +1,69 @@
+#!/bin/bash
+
+# exit when a command exits with an exit code != 0
+set -e
+
+# version is used by `docker-compose.yml` do determine the tag
+# of the Docker Image that is to be used
+export IMAGE="${IMAGE-netboxcommunity/netbox:latest}"
+
+if [ -z "${IMAGE}" ]; then
+  echo "⚠️ No image defined"
+
+  if [ -z "${DEBUG}" ]; then
+    exit 1;
+  else
+    echo "⚠️ Would 'exit 1' here, but DEBUG is '${DEBUG}'."
+  fi
+fi
+
+# The docker compose command to use
+doco="docker-compose -f docker-compose.test.yml"
+
+INITIALIZERS_DIR=".initializers"
+
+test_setup() {
+  echo "🏗 Setup up test environment"
+  if [ -d "${INITIALIZERS_DIR}" ]; then
+    rm -rf "${INITIALIZERS_DIR}"
+  fi
+
+  mkdir "${INITIALIZERS_DIR}"
+  (
+    cd initializers
+    for script in *.yml; do
+      sed -E 's/^# //' "${script}" > "../${INITIALIZERS_DIR}/${script}"
+    done
+  )
+}
+
+test_netbox_unit_tests() {
+  echo "⏱ Running Netbox Unit Tests"
+  $doco run --rm netbox ./manage.py test
+}
+
+test_initializers() {
+  echo "🏭 Testing Initializers"
+  export INITIALIZERS_DIR
+  $doco run --rm netbox ./manage.py check
+}
+
+test_cleanup() {
+  echo "💣 Cleaning Up"
+  $doco down -v
+
+  if [ -d "${INITIALIZERS_DIR}" ]; then
+    rm -rf "${INITIALIZERS_DIR}"
+  fi
+}
+
+echo "🐳🐳🐳 Start testing '${IMAGE}'"
+
+# Make sure the cleanup script is executed
+trap test_cleanup EXIT ERR
+test_setup
+
+test_netbox_unit_tests
+test_initializers
+
+echo "🐳🐳🐳 Done testing '${IMAGE}'"