docs: Move all docs

Signed-off-by: Julius Härtl <jus@bitgrid.net>
This commit is contained in:
Julius Härtl
2023-12-03 15:30:21 +01:00
parent e941833242
commit b777c14fa9
30 changed files with 422 additions and 452 deletions

5
.gitignore vendored
View File

@@ -8,3 +8,8 @@
/data/skeleton/
/wip/
/site
node_modules/
/tests/test-results/
/tests/playwright-report/
/tests/blob-report/
/tests/playwright/.cache/

View File

@@ -1,6 +1,6 @@
SHELL := /bin/bash
.PHONY: images docker-build pull-all
.PHONY: images docker-build pull-all docs docs-watch
.ONESHELL:
images: docker/*/Dockerfile docker/Dockerfile.*
@@ -55,3 +55,11 @@ template-apply:
cat docker/Dockerfile.php.template | sed 's/php:8.2/php:8.1/' > docker/Dockerfile.php81
cat docker/Dockerfile.php.template | sed 's/php:8.2/php:8.2/' > docker/php82/Dockerfile
cat docker/Dockerfile.php.template | sed 's/php:8.2/php:8.3/' > docker/php83/Dockerfile
docs:
pip3 install mkdocs
mkdocs
docs-watch:
pip3 install mkdocs
mkdocs serve

433
README.md
View File

@@ -1,19 +1,18 @@
# Nextcloud development environment on Docker Compose
Nextcloud development environment using docker-compose
Nextcloud development environment using docker-compose providing a large variety of services for Nextcloud server and app development and testing.
**DO NOT USE THIS IN PRODUCTION** Various settings in this setup are considered insecure and default passwords and secrets are used all over the place
Features
- Nextcloud
- 🔒 Nginx proxy with SSL termination
- 💾 MySQL
- 💡 Redis
- 👥 LDAP with example user data
- ✉ Mailhog
- 🚀 Blackfire
- 📄 Collabora
- ☁ Nextcloud containers for running multiple versions
- 🐘 Multiple PHP versions
- 🔒 Nginx proxy with optional SSL termination
- 🛢️ MySQL/PostgreSQL/MariaDB/SQLite/MaxScale, Redis cache
- 💾 Local or S3 primary storage
- 👥 LDAP with example user data, Keycloak
- ✉ Mailhog for testing mail sending
- 🚀 Blackfire, Xdebug for profiling and debugging
- 📄 Lots of integrating service containers: Collabora Online, Onlyoffice, Elasticsearch, ...
## Tutorial
@@ -33,11 +32,10 @@ To start the setup run the following commands to clone the repository and bootst
git clone https://github.com/juliushaertl/nextcloud-docker-dev
cd nextcloud-docker-dev
./bootstrap.sh
````
```
This may take some time depending on your internet connection speed.
Once done you can start the Nextcloud container using:
```bash
docker-compose up nextcloud
@@ -94,409 +92,8 @@ docker run --rm -p 8080:80 -e SERVER_BRANCH=v24.0.1 -v /tmp/server:/var/www/html
```
## More features
Note that all of the following content will soon get moved and probably reworked into the [nextcloud-docker-dev documentation](https://juliushaertl.github.io/nextcloud-docker-dev/).
You can find documentation for more advanced features in [nextcloud-docker-dev documentation](https://juliushaertl.github.io/nextcloud-docker-dev/) for example:
### Additional apps
To install additional apps add them to the bootstrap command:
```bash
./bootstrap.sh circles contacts
```
You can also do this after the initial bootstrap.
In this case it will clone the apps but not update the `.env` file.
If you want your apps to be installed in the Nextcloud instance by default
add them to the `NEXTCLOUD_AUTOINSTALL_APPS` variable in `.env`.
### Running different stable versions
The docker-compose file provides individual containers for stable Nextcloud releases. In order to run those you will need a checkout of the stable version server branch to your workspace directory. Using [git worktree](https://blog.juliushaertl.de/index.php/2018/01/24/how-to-checkout-multiple-git-branches-at-the-same-time/) makes it easy to have different branches checked out in parallel in separate directories.
```
cd workspace/server
git worktree add ../stable23 stable23
```
As in the `server` folder, the `3rdparty` submodule is needed:
```
cd ../stable23
git submodule update --init
```
The same can be done for `stable24`, `stable25`... and so on.
Git worktrees can also be used to have a checkout of an apps stable branch within the server stable directory.
```
cd workspace/server/apps-extra/text
git worktree add ../../../stable23/apps-extra/text stable23
```
The viewer app should also be added to the stable worktrees:
```
cd workspace/server/apps/viewer
git worktree add ../../../stable25/apps/viewer stable25
```
- Use `apps/` for required apps (like `viewer`)
- Use `apps-extra/` for apps that support only one specific nextcloud version (like `talk`)
- Use `apps-shared/` for apps that support multiple nextcloud versions as this directory is shared between all containers
### Setting the PHP version to be used
The Nextcloud instance is setup to run with PHP 7.2 by default.
If you wish to use a different version of PHP, set the `PHP_VERSION` `.env` variable.
The variable supports the following values:
- PHP 7.1: `71`
- PHP 7.2: `72`
- PHP 7.3: `73`
- PHP 7.4: `74`
- PHP 8.0: `80`
- PHP 8.1: `81`
## Running into errors
- If your setup isn't working and you can not figure out the reason why, running
`docker-compose down -v` will remove the relevant containers and volumes,
allowing you to run `docker-compose up` again from a clean slate.
- Sometimes it might help: `docker pull ghcr.io/juliushaertl/nextcloud-dev-php74:latest`
- In extreme cases, clean everything: `docker system prune --all`
- If you start your stable containers (not the master) and it wants to install Nextcloud even if it is not the first start, you may have removed the configuration with the last `docker-compose down` command. Try to use `docker-compose stop` instead or give the stable setup named values yourself.
## 🔒 Reverse Proxy
Used for SSL termination. To setup SSL support provide a proper `DOMAIN_SUFFIX` environment variable and put the certificates to `./data/ssl/` named by the domain name.
You might need to add the domains to your `/etc/hosts` file:
```
127.0.0.1 nextcloud.local
127.0.0.1 collabora.local
```
This is assuming you have set `DOMAIN_SUFFIX=.local`
To update the hosts file automatically you can use the `update-hosts` script:
```
./scripts/update-hosts
```
You can generate self-signed certificates using:
```
cd data/ssl
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout nextcloud.local.key -out nextcloud.local.crt
```
You can also override the default port used for HTTP and HTTPS bound on the host for the proxy by setting these environment variables in the `.env` file (don't forget to recreate the containers):
```
PROXY_PORT_HTTP=8080
PROXY_PORT_HTTPS=4443
```
### dnsmasq to resolve wildcard domains
Instead of adding the individual container domains to `/etc/hosts` a local dns server like dnsmasq can be used to resolve any domain ending with the configured `DOMAIN_SUFFIX` in `.env` to localhost.
For dnsmasq adding the following configuration would be sufficient for `DOMAIN_SUFFIX=.local`:
```
address=/.local/127.0.0.1
```
To run dnsmasq in a container, you can use the following example:
```
docker run --rm -it \
-e DMQ_DHCP_RANGES=" " \
-e DMQ_DHCP_DNS=" " \
-e DMQ_DHCP_GATEWAY=" " \
-e DMQ_DNS_ADDRESS="address=/.local/127.0.0.1" \
-p 53:53 \
-p 53:53/udp \
drpsychick/dnsmasq:latest
```
### Use DNS Service Discovery on MacOS
You can also use the `dns-sd` tool on MacOS to advertise the container domains on the network. This is especially useful if you try to connect from an iPhone or iPad, since those devices do not allow to edit the `/etc/hosts` file. Use the tool like this:
```
dns-sd -P nextcloud _http._tcp local 80 nextcloud.local 192.168.0.10
```
Be aware that since this is advertised in the local network, it is not recommended to use it in a network where multiple instances could be running. In this case you might want to change the `DOMAIN_SUFFIX` in `.env` to prevent any collision.
### Use valid certificates trusted by your system
* Install [mkcert](https://github.com/FiloSottile/mkcert)
* Go to `data/ssl`
* `mkcert -cert-file nextcloud.local.crt -key-file nextcloud.local.key nextcloud.local`
* `docker-compose restart proxy`
## ✉ Mail
Sending/receiving mail can be tested with [mailhog](https://github.com/mailhog/MailHog) which is available on ports 1025 (SMTP).
To use the webui, add `127.0.0.1 mail.local` to your `/etc/hosts` and open [mail.local](http://mail.local).
## 🚀 Blackfire
Blackfire needs to use a hostname/ip that is resolvable from within the Blackfire container. Their free version is [limited to local profiling](https://support.blackfire.io/troubleshooting/hack-edition-users-cannot-profile-non-local-http-applications) so we need to browse Nextcloud though its local docker IP or add the hostname to `/etc/hosts`.
By default the PHP module for Blackfire is disabled, but you can enable or disable this through the following script:
```
./scripts/php-mod-config nextcloud blackfire on
```
After that you can use Blackfire through the browser plugin or curl as described below.
### Using with curl
```
alias blackfire='docker-compose exec -e BLACKFIRE_CLIENT_ID=$BLACKFIRE_CLIENT_ID -e BLACKFIRE_CLIENT_TOKEN=$BLACKFIRE_CLIENT_TOKEN blackfire blackfire'
blackfire curl http://192.168.21.8/
```
## configuration
You can specify custom config.php values in data/shared/config.php. The file
needs to be created before starting the container, but can be edited on the fly
without needing to restart the container. Note that this config is shared
accross all nextcloud containers.
## Xdebug
Xdebug is shipped but disabled by default. It can be turned on by running:
```
./scripts/php-mod-config nextcloud xdebug.mode debug
```
### Debugging cron, occ or other command line scripts
```
docker compose exec nextcloud bash
# use this if you have configured path mapping in PHPstorm to match the server name configured
export PHP_IDE_CONFIG=serverName=localhost
sudo -E -u www-data php -dxdebug.mode=debug -dxdebug.client_host=host.docker.internal -dxdebug.start_with_request=yes -dxdebug.idekey=PHPSTORM occ
```
## 👥 LDAP
The LDAP sample data is based on https://github.com/rroemhild/docker-test-openldap and extended with randomly generated users/groups. For details see [data/ldap-generator/](https://github.com/juliushaertl/nextcloud-docker-dev/tree/master/data/ldap-generator). LDAP will be configured automatically if the ldap container is available during installation.
Example users are: `leela fry bender zoidberg hermes professor`. The password is the same as the uid.
Useful commands:
```
docker-compose exec ldap ldapsearch -H 'ldap://localhost' -D "cn=admin,dc=planetexpress,dc=com" -w admin -b "dc=planetexpress,dc=com" "(&(objectclass=inetOrgPerson)(description=*use*))"
```
## Collabora
- Make sure to have the Collabora hostname setup in your `/etc/hosts` file: `127.0.0.1 collabora.local`
- Clone, build and enable the [richdocuments](https://github.com/nextcloud/richdocuments) app
- Automatically enable for one of your containers (e.g. the main `nextcloud` one):
- Run `./scripts/enable-collabora nextcloud`
- Manual setup
- Start the Collabora Online server in addition to your other containers `docker-compose up -d collabora`
- Make sure you have the [richdocuments app](https://github.com/nextcloud/richdocuments) cloned to your `apps-extra` directory and built the frontend code of the app with `npm ci && npm run build`
- Enable the app and configure `collabora.local` in the Collabora settings inside of Nextcloud
## ONLYOFFICE
- Make sure to have the ONLYOFFICE hostname setup in your `/etc/hosts` file: `127.0.0.1 onlyoffice.local`
- Automatically enable for one of your containers (e.g. the main `nextcloud` one):
- Run `./scripts/enable-onlyoffice nextcloud`
- Manual setup
- Start the ONLYOFFICE server in addition to your other containers `docker-compose up -d onlyoffice`
- Clone https://github.com/ONLYOFFICE/onlyoffice-nextcloud into your apps directory
- Enable the app and configure `onlyoffice.local` in the ONLYOFFICE settings inside of Nextcloud
## Talk HPB
- Make sure to have the signaling hostname setup in your `/etc/hosts` file: `127.0.0.1 talk-signaling.local`
- Automatically enable for one of your containers (e.g. the main `nextcloud` one):
- Run `./scripts/enable-talk-hpb.sh nextcloud`
- Manual setup
- Start the talk signaling server and janus in addition to your other containers `docker-compose up -d talk-signaling talk-janus`
- Go to the admin settings of talk and add the signaling server (`http://talk-signaling.local` with shared secret `1234`)
## Antivirus
```bash
docker-compose up -d proxy nextcloud av
```
The [ClamAV](https://www.clamav.net/) antivirus will then be exposed as a daemon with host `nextav` and
port `3310`.
## SAML
```
docker-compose up -d proxy nextcloud saml
```
- uid mapping: `urn:oid:0.9.2342.19200300.100.1.1`
- idp entity id: `https://sso.local.dev.bitgrid.net/simplesaml/saml2/idp/metadata.php`
- Single Sign-On (SSO) service url: `https://sso.local.dev.bitgrid.net/simplesaml/saml2/idp/SSOService.php`
- single log out service url: `https://sso.local.dev.bitgrid.net/simplesaml/saml2/idp/SingleLogoutService.php`
- use certificate from `docker/configs/var-simplesamlphp/cert/example.org.crt`
```
-----BEGIN CERTIFICATE-----
MIICrDCCAhWgAwIBAgIUNtfnC2jE/rLdxHCs2th3WaYLryAwDQYJKoZIhvcNAQEL
BQAwaDELMAkGA1UEBhMCREUxCzAJBgNVBAgMAkJZMRIwEAYDVQQHDAlXdWVyemJ1
cmcxFDASBgNVBAoMC0V4YW1wbGUgb3JnMSIwIAYDVQQDDBlzc28ubG9jYWwuZGV2
LmJpdGdyaWQubmV0MB4XDTE5MDcwMzE0MjkzOFoXDTI5MDcwMjE0MjkzOFowaDEL
MAkGA1UEBhMCREUxCzAJBgNVBAgMAkJZMRIwEAYDVQQHDAlXdWVyemJ1cmcxFDAS
BgNVBAoMC0V4YW1wbGUgb3JnMSIwIAYDVQQDDBlzc28ubG9jYWwuZGV2LmJpdGdy
aWQubmV0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDHPZwU+dAc76yB6bOq
0AkP1y9g7aAi1vRtJ9GD4AEAsA3zjW1P60BYs92mvZwNWK6NxlJYw51xPak9QMk5
qRHaTdBkmq0a2mWYqh1AZNNgCII6/VnLcbEIgyoXB0CCfY+2vaavAmFsRwOMdeR9
HmtQQPlbTA4m5Y8jWGVs1qPtDQIDAQABo1MwUTAdBgNVHQ4EFgQUeZSoGKeN5uu5
K+n98o3wcitFYJ0wHwYDVR0jBBgwFoAUeZSoGKeN5uu5K+n98o3wcitFYJ0wDwYD
VR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOBgQA25X/Ke+5dw7up8gcF2BNQ
ggBcJs+SVKBmPwRcPQ8plgX4D/K8JJNT13HNlxTGDmb9elXEkzSjdJ+6Oa8n3IMe
vUUejXDXUBvlmmm+ImJVwwCn27cSfIYb/RoZPeKtned4SCzpbEO9H/75z3XSqAZS
Z1tiHzYOVtEs4UNGOtz1Jg==
-----END CERTIFICATE-----
```
- cn `urn:oid:2.5.4.3`
- email `urn:oid:0.9.2342.19200300.100.1.3`
### Environment-based SSO
A simple approach to test environment-based SSO with the `user_saml` app is to use Apache's basic auth with the following configuration:
```
<Location /login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
<Location /index.php/login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
<Location /index.php/apps/user_saml/saml/login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
<Location /apps/user_saml/saml/login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
```
## [Fulltextsearch](https://github.com/nextcloud/fulltextsearch)
```
docker-compose up -d elasticsearch elasticsearch-ui
```
- Address for configuring in Nextcloud: `http://elastic:elastic@elasticsearch:9200`
- Address to access Elasticsearch from outside: `http://elastic:elastic@elasticsearch.local`
- Address for accessing the UI: http://elasticsearch-ui.local/
`sudo sysctl -w vm.max_map_count=262144`
## Object storage
Primary object storage can be enabled by setting the `PRIMARY=minio` environment variable either in your `.env` file or in `docker-compose.yml` for individual containers.
```bash
docker-compose up proxy nextcloud minio
```
## Development
### OCC
Run inside of the Nextcloud container:
```
set XDEBUG_CONFIG=idekey=PHPSTORM
sudo -E -u www-data php -dxdebug.remote_host=192.168.21.1 occ
```
### Useful commands
- Restart Apache to reload php configuration without a full container restart: `docker-compose kill -s USR1 nextcloud`
- Access to MySQL console: `mysql -h $(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nextcloud_database-mysql_1) -P 3306 -u nextcloud -pnextcloud`
- Run an LDAP search: `ldapsearch -x -H ldap://$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nextcloud_ldap_1) -D "cn=admin,dc=planetexpress,dc=com" -w admin -b "dc=planetexpress,dc=com" -s subtree <filter> <attrs>`
## [Keycloak](https://www.keycloak.org/)
- Keycloak is using LDAP as a user backend (make sure the LDAP container is also running)
- `occ user_oidc:provider Keycloak -c nextcloud -s 09e3c268-d8bc-42f1-b7c6-74d307ef5fde -d http://keycloak.dev.local/auth/realms/Example/.well-known/openid-configuration`
- http://keycloak.dev.local/auth/realms/Example/.well-known/openid-configuration
- nextcloud
- 09e3c268-d8bc-42f1-b7c6-74d307ef5fde
## Global scale
```
docker-compose up -d proxy portal gs1 gs2 lookup database-mysql
```
Users are named the same as the instance name, e.g. `gs1`, `gs2`
## Imaginary
Enable the imaginary server for generating previews
```bash
docker-compose up proxy nextcloud previews_hpb
./scripts/enable-preview-imaginary.sh
```
## PhpMyAdmin
If you need to access the database, you can startup the `phpmyadmin` container that is already prepared.
```
docker-compose up -d phpmyadmin
```
Just add the domain to your `/etc/hosts` file and give it a try.
```
sudo sh -c "echo '127.0.0.1 phpmyadmin.local' >> /etc/hosts"
```
## pgAdmin
If you need to access the database and you are running PostgreSQL, you can use this additional container.
```
docker-compose up -d pgadmin
```
Add the domain to your `/etc/hosts` file:
```
sudo sh -c "echo '127.0.0.1 pgadmin.local' >> /etc/hosts"
```
After you have started the container open `pgadmin.local` in a web browser. The password for the `nextcloud.local` is `postgres`.
That's it, open the following path to see the Nextcloud tables: `Server group 1 -> nextcloud.local -> Databases -> nextcloud -> Schemas -> public -> Tables`
- Running stable Nextcloud versions in parallel
- Using different database backends
- Using HTTPS

View File

@@ -1,21 +1,77 @@
# Getting started
The easiest way to get the setup running the ```master``` branch is by running the ```bootstrap.sh``` script:
```
## Tutorial
You can find a [step by step tutorial on how to use this setup](https://cloud.nextcloud.com/s/iyNGp8ryWxc7Efa?path=%2F1%20Setting%20up%20a%20development%20environment) in the [Nextcloud Developer Portal](https://nextcloud.com/developer/). It will guide you through the setup and show you how to use it for app development.
There are two ways of using this setup. **A persistent setup is the most common one** where you have a local git clone of all required repositories. The other one is a standalone setup that can be used to quickly run parts of the Nextcloud ecosystem source code without having to clone everything.
## Persistent development setup
!!! note
This is the recommended way to run the setup for development. You will have a local clone of all required source code.
To start the setup run the following commands to clone the repository and bootstrap the setup. This will prepare your setp and clone the Nextcloud server repository and required apps into the `workspace` folder.
```bash
git clone https://github.com/juliushaertl/nextcloud-docker-dev
cd nextcloud-docker-dev
./bootstrap.sh
sudo sh -c "echo '127.0.0.1 nextcloud.local' >> /etc/hosts"
docker-compose up nextcloud proxy
```
This will clone the server repository into the ```workspace/server``` directory and start the containers. The server will be available at http://nextcloud.local.
This may take some time depending on your internet connection speed.
Note that for performance reasons the server repository might have been cloned with --depth=1 by default. To get the full history it is highly recommended to run:
cd workspace/server
git fetch --unshallow
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
git fetch origin
Once done you can start the Nextcloud container using:
```bash
docker-compose up nextcloud
```
This may take some time depending on your internet connection speed.
You can also start it in the background using `docker-compose up -d nextcloud`.
You can then access your Nextcloud instance at [http://nextcloud.local](http://nextcloud.local). The default username is `admin` and the password is `admin`. [Other users can be found in the documentation](https://juliushaertl.github.io/nextcloud-docker-dev/basics/overview/#default-users).
!!! warning
Note that for performance reasons the server repository might have been cloned with `--depth=1` by default. To get the full history it is highly recommended to run:
```bash
cd workspace/server
git fetch --unshallow
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
git fetch origin
```
## Standalone containers
!!! note
This is a very simple way but doesn't cover all features. If you are looking for a fully featured setup you may skip to the next section
There is a standalone version of the Nextcloud containers available that can be used to run Nextcloud without the other services. This is useful if you are just wanting to get started with app development against a specific server version, or to just have a quick way to develop, test or debug.
These containers support automatic fetching of the server source code and use SQLite as the database. The server source code is fetched from the official Nextcloud server repository and the version can be specified using the `NEXTCLOUD_VERSION` environment variable. The default version is `master`.
Running the containers does not need this repository to be cloned.
Example for running a Nextcloud server from the master branch of server:
```bash
docker run --rm -p 8080:80 ghcr.io/juliushaertl/nextcloud-dev-php80:latest
```
For app development you can mount your app directly into the container:
```bash
docker run --rm -p 8080:80 -v ~/path/to/appid:/var/www/html/apps-extra/appid ghcr.io/juliushaertl/nextcloud-dev-php80:latest
```
The `SERVER_BRANCH` environment variable can be used to run different versions of Nextcloud by specificing either a server branch or git tag.
```bash
docker run --rm -p 8080:80 -e SERVER_BRANCH=v24.0.1 ghcr.io/juliushaertl/nextcloud-dev-php80:latest
```
You can also mount your local server source code into the container to run a local version of Nextcloud:
```bash
docker run --rm -p 8080:80 -e SERVER_BRANCH=v24.0.1 -v /tmp/server:/var/www/html ghcr.io/juliushaertl/nextcloud-dev-php80:latest
```

View File

@@ -15,6 +15,7 @@ You can do this by running the following command, which will automatically updat
Instead of adding the individual container domains to `/etc/hosts` a local dns server like dnsmasq can be used to resolve any domain ending with the configured `DOMAIN_SUFFIX` in `.env` to localhost.
For dnsmasq adding the following configuration would be sufficient for `DOMAIN_SUFFIX=.local`:
```
address=/.local/127.0.0.1
```
@@ -40,4 +41,4 @@ You can also use the `dns-sd` tool on MacOS to advertise the container domains o
dns-sd -P nextcloud _http._tcp local 80 nextcloud.local 192.168.0.10
```
Be aware that since this is advertised in the local network, it is not recommended to use it in a network where multiple instances could be running. In this case you might want to change the `DOMAIN_SUFFIX` in `.env` to prevent any collision.
Be aware that since this is advertised in the local network, it is not recommended to use it in a network where multiple instances could be running. In this case you might want to change the `DOMAIN_SUFFIX` in `.env` to prevent any collision.

View File

@@ -18,4 +18,30 @@ The following user accounts are available by default:
- `user1` / `user1`
- `user2` / `user2`
- `user3` / `user3`
- `user4` / `user4`
- `user4` / `user4`
## App directories
The Nextcloud containers are configured to use multiple app directories.
- Use `apps/` for required apps (like `viewer`)
- Use `apps-extra/` for apps that support only one specific nextcloud version (like `talk`)
- Use `apps-shared/` for apps that support multiple nextcloud versions as this directory is shared between all containers
## Cronjobs
The cronjobs are configured to run every 5 minutes in the individual containers.
For testing you can also run them manually:
```bash
docker-compose exec nextcloud php cron.php
```
### occ
Run inside of the Nextcloud container:
```
set XDEBUG_CONFIG=idekey=PHPSTORM
sudo -E -u www-data php -dxdebug.remote_host=192.168.21.1 occ

View File

@@ -1,9 +1,23 @@
# HTTPS
# Use valid certificates trusted by your system
A nginx proxy container is used to route requests to the correct container. This proxy is automatically started. It can be configured to use HTTPS by setting the `PROTOCOL` environment variable to `https` in the `.env` file. The container will pick up SSL certificates automatically from `data/ssl/` named by the domain name.
# Use mkcert
mkcert is a simple tool for making locally-trusted development certificates. It requires no configuration. This would be the recommended way to generate certificates for local development.
* Install [mkcert](https://github.com/FiloSottile/mkcert)
* Go to `data/ssl`
* `mkcert -cert-file nextcloud.local.crt -key-file nextcloud.local.key nextcloud.local`
* Add `PROTOCOL=https` to your `.env` file
* `docker-compose restart proxy`
* There is also a script to generate/update all certs: `./scripts/update-certs`
* There is also a script to generate/update all certs: `./scripts/update-certs`
## Use self-signed certificates
You can generate self-signed certificates using:
```
cd data/ssl
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout nextcloud.local.key -out nextcloud.local.crt
```

View File

@@ -0,0 +1,37 @@
# Stable Nextcloud versions
As described in the [overview](overview.md) there are multiple Nextcloud containers available. The main `nextcloud` container is targetting the main workspace directly (usually for running the master/main branch of Nextcloud server and apps) of the latest development version. In addition there are stable containers for running the stable major version branches in parallel.
## Prepare your git checkouts for running a stable version
In order to run a stable version you need to have the corresponding git checkouts available. Using [git worktree](https://blog.juliushaertl.de/index.php/2018/01/24/how-to-checkout-multiple-git-branches-at-the-same-time/) makes it easy to have different branches checked out in parallel in separate directories and is the recommended way to work with stable branches in parallel.
Assuimg you have already cloned the repository into `~/nextcloud-docker-dev/workspace/server` you can run the following commands to create a new worktree for the stable28 branch:
```bash
# create a new worktree for the stable28 branch
cd ~/nextcloud-docker-dev/workspace/server
git worktree add ../stable28 stable28
cd ~/nextcloud-docker-dev/workspace/stable28
# make sure submodules are installed in the stable server root directory
git submodule update --init
```
### Add worktree for additional apps
This will be required for every app that you need on the stable branches, so run this for viewer but also for any other app you need.
```bash
cd ~/nextcloud-docker-dev/workspace/server/apps/viewer
git worktree add ../../../stable28/apps/viewer stable28
```
## Start the stable28 container
```bash
docker-compose up -d stable28
```
## Apps without stable branches
Some apps do not have stable branches or cover multiple Nextcloud version. You can use the `ADDITIONAL_APPS_PATH` variable in your `.env` file to add add a cloned app to all Nextcloud containers. By default this is set to `./data/apps-extra`

View File

@@ -30,4 +30,4 @@ You can also mount your local server source code into the container to run a loc
```bash
docker run --rm -p 8080:80 -e SERVER_BRANCH=v24.0.1 -v /tmp/server:/var/www/html ghcr.io/juliushaertl/nextcloud-dev-php80:latest
```
```

View File

@@ -0,0 +1,13 @@
# Troubleshooting
- If your setup isn't working and you can not figure out the reason why, running
`docker-compose down -v` will remove the relevant containers and volumes,
allowing you to run `docker-compose up` again from a clean slate.
- You can run `./bootstrap.sh` again to check the setup and detected paths for your source code destination
- In extreme cases, clean everything: `docker system prune --all`
- If you start your stable containers (not the master) and it wants to install Nextcloud even if it is not the first start, you may have removed the configuration with the last `docker-compose down` command. Try to use `docker-compose stop` instead or give the stable setup named values yourself.
## Logs
- You can use `docker-compose logs -f` to follow the logs of all containers
- You can use `docker-compose logs -f nextcloud` to follow the logs of the Nextcloud container

22
docs/basics/update.md Normal file
View File

@@ -0,0 +1,22 @@
# Update
## Updating the development environment
- `git pull` to get the latest changes
- `make pull-installed` to pull the latest versions of all images that are already downloaded
- `make pull-all` to pull the latest versions of all images
- After pulling make sure to either recreate the containers manually or recreate the full development environment through `docker compose down -v` and `docker compose up -d proxy nextcloud ...` for the containers in use.
## Updating the Nextcloud server
As Nextcloud containers are bound to a server major version and the code is updated through manual git pull, you only need to call occ update on demand
```bash
docker-compose exec nextcloud occ upgrade
```
### Major version bump
In case Nextcloud server bumps the major version, you will need to pull all repositories again to the latest state to get the compatibility changes.
You might want to take the opportunity to then setup the previous version as a new stable version setup. See [stable versions](stable-versions.md) for more information.

13
docs/customizing/build.md Normal file
View File

@@ -0,0 +1,13 @@
# Build containers
This is usually only required if you want to test changes to the containers or if you want to build the containers yourself instead of using the prebuilt images.
You can build the containers manually for testing local changes by calling make with the Dockerfile as the target:
```bash
make docker/php82/Dockerfile
make docker/Dockerfile.php81
make docker/Dockerfile.php80
```
Afterwards you can recrate the container with `docker-compose up -d --force-recreate nextcloud` to run the new image or use `docker-compose down -v` before to also reinstall Nextcloud.

View File

@@ -1,5 +1,25 @@
# Config
## PHP Version
The PHP version can be changed by setting the `PHP_VERSION` environment variable in your local `.env` file. If no value is set the minimum required version for the current Nextcloud version will be used depending on the Nextcloud container.
```bash
# For using PHP 8.3
PHP_VERSION=83
```
The variable supports the following values:
- PHP 7.1: `71`
- PHP 7.2: `72`
- PHP 7.3: `73`
- PHP 7.4: `74`
- PHP 8.0: `80`
- PHP 8.1: `81`
- PHP 8.2: `82`
- PHP 8.3: `83` (currently the xdebug and imagick php extensions are not available for this version)
## Nextcloud config.php
The config.php file of Nextcloud is pre-seeded with lots of configuration values. In order to change them you can place a personal config.php file in `data/shared/config.php`. This file will be included after the default config.php file for all Nextcloud containers.
@@ -13,4 +33,4 @@ $CONFIG = [
'apps' => ['myapp'],
],
];
```
```

View File

@@ -3,6 +3,7 @@
In order to customize the behavior of the application, you can use hooks. Hooks need to be placed in the `data/shared/hooks/` directory. They will be picked up by the docker containers automatically. They can be used for automating setup specific to a developers use cases. For example, you can use them to create a user, install an app, or run a script before or after the installation of Nextcloud.
The following hooks are currently available:
- before-install.sh Runs before the installation of Nextcloud
- after-install.sh Runs after the installation of Nextcloud
- before-start.sh Runs before the start of Nextcloud webserver
@@ -25,4 +26,4 @@ occ user:add --password-from-env mycustomuser
echo 'Always disable the firstrunwizard'
occ app:disable firstrunwizard
```
```

View File

@@ -10,6 +10,6 @@ The following shell function allows you to run `nc-dev` instead of `docker-compo
````
nc-dev() {
(cd ~/path/to/nextcloud-docker-dev && docker-compose $@)
(cd ~/path/to/nextcloud-docker-dev && docker-compose $@)
}
```
```

View File

@@ -1,3 +1,17 @@
# Nextcloud docker development environment
This documentation covers a Nextcloud development environment using docker-compose providing a large variety of services for Nextcloud server and app development and testing.
**DO NOT USE THIS IN PRODUCTION**
Various settings in this setup are considered insecure and default passwords and secrets are used all over the place
- ☁ Nextcloud containers for running multiple versions
- 🐘 Multiple PHP versions
- 🔒 Nginx proxy with optional SSL termination
- 🛢️ MySQL/PostgreSQL/MariaDB/SQLite/MaxScale, Redis cache
- 💾 Local or S3 primary storage
- 👥 LDAP with example user data, Keycloak
- ✉ Mailhog for testing mail sending
- 🚀 Blackfire, Xdebug for profiling and debugging
- 📄 Lots of integrating service containers: Collabora Online, Onlyoffice, Elasticsearch, ...
Follow the [getting started guide](https://juliushaertl.github.io/nextcloud-docker-dev/basics/getting-started/) or the [Nextcloud developer tutorial](https://nextcloud.com/developer/) to get started.

View File

@@ -5,4 +5,4 @@ docker-compose up -d proxy nextcloud av
```
The [ClamAV](https://www.clamav.net/) antivirus will then be exposed as a daemon with host `nextav` and
port `3310`.
port `3310`.

View File

@@ -13,12 +13,12 @@ By default MySQL will be used as database backend. You can change this by settin
Changing the database env value will require to recreate your setup. You can do this by running `docker-compose down -v` and then `docker-compose up -d nextcloud`.
All databases use the following credentials by default:
- Root password: `nextcloud`
- Username: `nextcloud`
- Password: `nextcloud`
- Database: `nextcloud` or the name of the stable container e.g. `stable27``
## Accessing the database
### MySQL/MariaDB
@@ -29,7 +29,7 @@ You can access the database with the following command:
docker-compose exec mariadb mysql -u root -pnextcloud
```
If you prefer a GUI frontend you can additionally launch the phpmyadmin container with `docker-compose up -d phpmyadmin` and access it via http://phpmyadmin.local.
If you prefer a GUI frontend you can additionally launch the phpmyadmin container with `docker-compose up -d phpmyadmin` and access it via <http://phpmyadmin.local>.
Alternatively you can use a database client to access the database from the host system. The port can be obtained with `docker-compose port database-mysql 3306`. The host is `localhost` and the credentials are the same as above.
@@ -41,10 +41,13 @@ You can access the database with the following command:
docker-compose exec postgres psql -U nextcloud -d nextcloud
```
If you prefer a GUI frontend you can additionally launch the pgadmin container with `docker-compose up -d pgadmin` and access it via http://pgadmin.local.
If you prefer a GUI frontend you can additionally launch the pgadmin container with `docker-compose up -d pgadmin` and access it via <http://pgadmin.local>.
Alternatively you can use a database client to access the database from the host system. The port can be obtained with `docker-compose port database-postgresql 5432`. The host is `localhost` and the credentials are the same as above.
After you have started the container open `pgadmin.local` in a web browser. The password for the `nextcloud.local` is `postgres`.
That's it, open the following path to see the Nextcloud tables: `Server group 1 -> nextcloud.local -> Databases -> nextcloud -> Schemas -> public -> Tables`
### SQLite
You can access the database with the following command:
@@ -68,4 +71,4 @@ docker-compose exec database-mariadb-replica mysql -u root -pnextcloud
This mode runs a mariadb primary and read replica setup with maxscale as load balancer. The primary is used for writes and the replica for reads where MaxScale is used to perform a read-write-split.
The logs of MaxScale can be accessed with `docker-compose exec maxscale cat /var/log/maxscale/maxscale.log`.
The logs of MaxScale can be accessed with `docker-compose exec maxscale cat /var/log/maxscale/maxscale.log`.

View File

@@ -0,0 +1,8 @@
# Imaginary
Enable the imaginary server for generating previews
```bash
docker-compose up proxy nextcloud previews_hpb
./scripts/enable-preview-imaginary.sh
```

View File

@@ -1,6 +1,6 @@
# LDAP
The LDAP sample data is based on https://github.com/rroemhild/docker-test-openldap and extended with randomly generated users/groups. For details see [data/ldap-generator/](https://github.com/juliushaertl/nextcloud-docker-dev/tree/master/data/ldap-generator). LDAP will be configured automatically if the ldap container is available during installation.
The LDAP sample data is based on <https://github.com/rroemhild/docker-test-openldap> and extended with randomly generated users/groups. For details see [data/ldap-generator/](https://github.com/juliushaertl/nextcloud-docker-dev/tree/master/data/ldap-generator). LDAP will be configured automatically if the ldap container is available during installation.
Example users are: `leela fry bender zoidberg hermes professor`. The password is the same as the uid.
@@ -9,4 +9,3 @@ Example users are: `leela fry bender zoidberg hermes professor`. The password is
- Run an LDAP search
- `ldapsearch -x -H ldap://$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' nextcloud_ldap_1) -D "cn=admin,dc=planetexpress,dc=com" -w admin -b "dc=planetexpress,dc=com" -s subtree <filter> <attrs>`
- `docker-compose exec ldap ldapsearch -H 'ldap://localhost' -D "cn=admin,dc=planetexpress,dc=com" -w admin -b "dc=planetexpress,dc=com" "(&(objectclass=inetOrgPerson)(description=*use*))"`

View File

@@ -1,3 +1,3 @@
# Mail
Sending mails from Nextcloud usually requires an email server and account to be configured. This setup provides a [mailhog](https://github.com/mailhog/MailHog) installation that is autoconfigured, so you can browse all mails sent from Nextcloud in the webui by opening [mail.local](http://mail.local).
Sending mails from Nextcloud usually requires an email server and account to be configured. This setup provides a [mailhog](https://github.com/mailhog/MailHog) installation that is autoconfigured, so you can browse all mails sent from Nextcloud in the webui by opening [mail.local](http://mail.local) in your browser.

19
docs/services/more.md Normal file
View File

@@ -0,0 +1,19 @@
## Global scale
```
docker-compose up -d proxy portal gs1 gs2 lookup database-mysql
```
Users are named the same as the instance name, e.g. `gs1`, `gs2`
## [Fulltextsearch](https://github.com/nextcloud/fulltextsearch)
```
docker-compose up -d elasticsearch elasticsearch-ui
```
- Address for configuring in Nextcloud: `http://elastic:elastic@elasticsearch:9200`
- Address to access Elasticsearch from outside: `http://elastic:elastic@elasticsearch.local`
- Address for accessing the UI: <http://elasticsearch-ui.local/>
`sudo sysctl -w vm.max_map_count=262144`

View File

@@ -26,4 +26,4 @@ To properly work with HTTPS, you need to add the following parameter to the Coll
```
COLLABORA_PARAMS="--o:ssl.termination=true"
```
```

View File

@@ -16,5 +16,5 @@ It requires to have the onlyoffice integration app cloned into your apps directo
- Make sure to have the ONLYOFFICE hostname setup in your `/etc/hosts` file: `127.0.0.1 onlyoffice.local`
- Start the ONLYOFFICE server in addition to your other containers `docker-compose up -d onlyoffice`
- Clone https://github.com/ONLYOFFICE/onlyoffice-nextcloud into your apps directory
- Enable the app and configure `onlyoffice.local` in the ONLYOFFICE settings inside of Nextcloud
- Clone <https://github.com/ONLYOFFICE/onlyoffice-nextcloud> into your apps directory
- Enable the app and configure `onlyoffice.local` in the ONLYOFFICE settings inside of Nextcloud

9
docs/services/s3.md Normal file
View File

@@ -0,0 +1,9 @@
# S3 / Object storage
## Primary object storage
Primary object storage can be enabled by setting the `PRIMARY=minio` environment variable either in your `.env` file or in `docker-compose.yml` for individual containers.
```bash
docker-compose up nextcloud minio
```

77
docs/services/sso.md Normal file
View File

@@ -0,0 +1,77 @@
# SSO/SAML/OpenID Connect
## [Keycloak](https://www.keycloak.org/)
- Keycloak is using LDAP as a user backend (make sure the LDAP container is also running)
- `occ user_oidc:provider Keycloak -c nextcloud -s 09e3c268-d8bc-42f1-b7c6-74d307ef5fde -d http://keycloak.dev.local/auth/realms/Example/.well-known/openid-configuration`
- <http://keycloak.dev.local/auth/realms/Example/.well-known/openid-configuration>
- nextcloud
- 09e3c268-d8bc-42f1-b7c6-74d307ef5fde
## SAML
```
docker-compose up -d proxy nextcloud saml
```
- uid mapping: `urn:oid:0.9.2342.19200300.100.1.1`
- idp entity id: `https://sso.local.dev.bitgrid.net/simplesaml/saml2/idp/metadata.php`
- Single Sign-On (SSO) service url: `https://sso.local.dev.bitgrid.net/simplesaml/saml2/idp/SSOService.php`
- single log out service url: `https://sso.local.dev.bitgrid.net/simplesaml/saml2/idp/SingleLogoutService.php`
- use certificate from `docker/configs/var-simplesamlphp/cert/example.org.crt`
```
-----BEGIN CERTIFICATE-----
MIICrDCCAhWgAwIBAgIUNtfnC2jE/rLdxHCs2th3WaYLryAwDQYJKoZIhvcNAQEL
BQAwaDELMAkGA1UEBhMCREUxCzAJBgNVBAgMAkJZMRIwEAYDVQQHDAlXdWVyemJ1
cmcxFDASBgNVBAoMC0V4YW1wbGUgb3JnMSIwIAYDVQQDDBlzc28ubG9jYWwuZGV2
LmJpdGdyaWQubmV0MB4XDTE5MDcwMzE0MjkzOFoXDTI5MDcwMjE0MjkzOFowaDEL
MAkGA1UEBhMCREUxCzAJBgNVBAgMAkJZMRIwEAYDVQQHDAlXdWVyemJ1cmcxFDAS
BgNVBAoMC0V4YW1wbGUgb3JnMSIwIAYDVQQDDBlzc28ubG9jYWwuZGV2LmJpdGdy
aWQubmV0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDHPZwU+dAc76yB6bOq
0AkP1y9g7aAi1vRtJ9GD4AEAsA3zjW1P60BYs92mvZwNWK6NxlJYw51xPak9QMk5
qRHaTdBkmq0a2mWYqh1AZNNgCII6/VnLcbEIgyoXB0CCfY+2vaavAmFsRwOMdeR9
HmtQQPlbTA4m5Y8jWGVs1qPtDQIDAQABo1MwUTAdBgNVHQ4EFgQUeZSoGKeN5uu5
K+n98o3wcitFYJ0wHwYDVR0jBBgwFoAUeZSoGKeN5uu5K+n98o3wcitFYJ0wDwYD
VR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOBgQA25X/Ke+5dw7up8gcF2BNQ
ggBcJs+SVKBmPwRcPQ8plgX4D/K8JJNT13HNlxTGDmb9elXEkzSjdJ+6Oa8n3IMe
vUUejXDXUBvlmmm+ImJVwwCn27cSfIYb/RoZPeKtned4SCzpbEO9H/75z3XSqAZS
Z1tiHzYOVtEs4UNGOtz1Jg==
-----END CERTIFICATE-----
```
- cn `urn:oid:2.5.4.3`
- email `urn:oid:0.9.2342.19200300.100.1.3`
## Environment-based SSO
A simple approach to test environment-based SSO with the `user_saml` app is to use Apache's basic auth with the following configuration:
```
<Location /login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
<Location /index.php/login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
<Location /index.php/apps/user_saml/saml/login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
<Location /apps/user_saml/saml/login>
AuthType Basic
AuthName "SAML"
AuthUserFile /var/www/html/.htpasswd
Require valid-user
</Location>
```

8
docs/services/talk.md Normal file
View File

@@ -0,0 +1,8 @@
# Talk HPB
- Make sure to have the signaling hostname setup in your `/etc/hosts` file: `127.0.0.1 talk-signaling.local`
- Automatically enable for one of your containers (e.g. the main `nextcloud` one):
- Run `./scripts/enable-talk-hpb.sh nextcloud`
- Manual setup
- Start the talk signaling server and janus in addition to your other containers `docker-compose up -d talk-signaling talk-janus`
- Go to the admin settings of talk and add the signaling server (`http://talk-signaling.local` with shared secret `1234`)

18
docs/tools/blackfire.md Normal file
View File

@@ -0,0 +1,18 @@
# Blackfire
Blackfire needs to use a hostname/ip that is resolvable from within the Blackfire container. Their free version is [limited to local profiling](https://support.blackfire.io/troubleshooting/hack-edition-users-cannot-profile-non-local-http-applications) so we need to browse Nextcloud though its local docker IP or add the hostname to `/etc/hosts`.
By default the PHP module for Blackfire is disabled, but you can enable or disable this through the following script:
```
./scripts/php-mod-config nextcloud blackfire on
```
After that you can use Blackfire through the browser plugin or curl as described below.
### Using with curl
```
alias blackfire='docker-compose exec -e BLACKFIRE_CLIENT_ID=$BLACKFIRE_CLIENT_ID -e BLACKFIRE_CLIENT_TOKEN=$BLACKFIRE_CLIENT_TOKEN blackfire blackfire'
blackfire curl http://192.168.21.8/
```

View File

@@ -19,4 +19,4 @@ docker compose exec nextcloud bash
# use this if you have configured path mapping in PHPstorm to match the server name configured
export PHP_IDE_CONFIG=serverName=localhost
sudo -E -u www-data php -dxdebug.mode=debug -dxdebug.client_host=host.docker.internal -dxdebug.start_with_request=yes -dxdebug.idekey=PHPSTORM occ
```
```

View File

@@ -1,3 +1,5 @@
site_name: nextcloud-docker-dev
theme:
name: readthedocs
markdown_extensions:
- admonition