Could not connect to database, check logs/librenms.log. Need some help

Korvo · April 7, 2023, 8:58pm

Hi everyone,

I have some trouble getting LibreNMS and mariadb to work on Synology Docker following this tutorial:
Synology SNMP Network Monitoring with LibreNMS Docker - Jason Loong | Tech Enthusiast, Photographer & Drummer

At the moment I get to setup: docker exec librenms setup_database I receive an error message:

Could not connect to database, check logs/librenms.log

This is the librenms log file:

[2023-04-07T20:07:35.154979+02:00] local.ERROR: PDOException: SQLSTATE[HY000] [1045] Access denied for user ‘librenms’@‘172.17.0.13’ (using password: YES) in /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:40
Stack trace:
[2023-04-07T22:37:47.122224+02:00] production.ERROR: PDOException: SQLSTATE[HY000] [1045] Access denied for user ‘librenms’@‘172.17.0.13’ (using password: YES) in /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:40
Stack trace:
#0 /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php(40): PDO->__construct()
#1 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php(67): Doctrine\DBAL\Driver\PDOConnection->__construct()
#2 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php(46): Illuminate\Database\Connectors\Connector->createPdoConnection()
#3 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/MySqlConnector.php(24): Illuminate\Database\Connectors\Connector->createConnection()
#4 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/ConnectionFactory.php(184): Illuminate\Database\Connectors\MySqlConnector->connect()
#5 [internal function]: Illuminate\Database\Connectors\ConnectionFactory->Illuminate\Database\Connectors{closure}()
#6 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(1064): call_user_func()
#7 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(1100): Illuminate\Database\Connection->getPdo()
#8 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(442): Illuminate\Database\Connection->getReadPdo()
#9 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(368): Illuminate\Database\Connection->getPdoForSelect()
#10 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(705): Illuminate\Database\Connection->Illuminate\Database{closure}()
#11 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(672): Illuminate\Database\Connection->runQueryCallback()
#12 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connection.php(376): Illuminate\Database\Connection->run()
#13 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2414): Illuminate\Database\Connection->select()
#14 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2402): Illuminate\Database\Query\Builder->runSelect()
#15 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2936): Illuminate\Database\Query\Builder->Illuminate\Database\Query{closure}()
#16 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2403): Illuminate\Database\Query\Builder->onceWithColumns()
#17 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(625): Illuminate\Database\Query\Builder->get()
#18 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(609): Illuminate\Database\Eloquent\Builder->getModels()
#19 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Concerns/BuildsQueries.php(294): Illuminate\Database\Eloquent\Builder->get()
#20 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(474): Illuminate\Database\Eloquent\Builder->first()
#21 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(508): Illuminate\Database\Eloquent\Builder->firstOrNew()
#22 /opt/librenms/vendor/laravel/framework/src/Illuminate/Support/Traits/ForwardsCalls.php(23): Illuminate\Database\Eloquent\Builder->updateOrCreate()
#23 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2132): Illuminate\Database\Eloquent\Model->forwardCallTo()
#24 /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php(2144): Illuminate\Database\Eloquent\Model->__call()
#25 /opt/librenms/LibreNMS/Config.php(262): Illuminate\Database\Eloquent\Model::__callStatic()
#26 /opt/librenms/LibreNMS/Config.php(460): LibreNMS\Config::persist()
#27 /opt/librenms/LibreNMS/Config.php(61): LibreNMS\Config::processConfig()
#28 /opt/librenms/app/Providers/ConfigServiceProvider.php(27): LibreNMS\Config::load()
#29 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): App\Providers\ConfigServiceProvider->boot()
#30 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container{closure}()
#31 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#32 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(37): Illuminate\Container\BoundMethod::callBoundMethod()
#33 /opt/librenms/vendor/laravel/framework/src/Illuminate/Container/Container.php(653): Illuminate\Container\BoundMethod::call()
#34 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(924): Illuminate\Container\Container->call()
#35 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(905): Illuminate\Foundation\Application->bootProvider()
#36 [internal function]: Illuminate\Foundation\Application->Illuminate\Foundation{closure}()
#37 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(906): array_walk()
#38 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/BootProviders.php(17): Illuminate\Foundation\Application->boot()
#39 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(237): Illuminate\Foundation\Bootstrap\BootProviders->bootstrap()
#40 /opt/librenms/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(310): Illuminate\Foundation\Application->bootstrapWith()
#41 /opt/librenms/LibreNMS/Util/Laravel.php(44): Illuminate\Foundation\Console\Kernel->bootstrap()
#42 /opt/librenms/includes/init.php(88): LibreNMS\Util\Laravel::bootCli()
#43 /opt/librenms/includes/sql-schema/update.php(30): require(‘…’)
#44 {main}

My mariadb container is on IP 172.17.0.12
My LibreNMS container IP 172.17.0.13
My Synology NAS IP 192.168.1.202

I don’t use the firewall on the Synology NAS.

This is the mariadb docker setup I use:
docker run -d --name=mariadb
-p 3306:3306
-e PUID=1027
-e PGID=100
-e TZ=Europe/Brussels
-e MYSQL_ROOT_PASSWORD=SomeSecretPassword
-v /volume1/docker/mariadb:/etc/mysql/conf.d
–sql-mode=’’
–restart always
mariadb:latest

Using this two additional line give errors:

--innodb_file_per_table=1 \
--lower_case_table_names=0 \

I guess these enviroment settings schould defined as ‘-e’ be but not tested yet:

-e innodb_file_per_table=1 \
-e lower_case_table_names=0 \

This is my LibreNMS Docket setting:

docker run -d --name librenms
-p 8668:80
-e DB_HOST=libredb
-e DB_NAME=librenms
-e DB_USER=librenms
-e DB_PASS=SomeSecretPassword
-e APP_KEY=base64: PUT YOUR GENERATED APP KEY HERE
-e BASE_URL=http://192.168.1.202:8668
-e TZ=Europe/Brussels
–link mariadb:libredb
-v /volume1/docker/librenms/logs:/opt/librenms/logs
-v /volume1/docker/librenms/rrd:/opt/librenms/rrd
-v /volume1/docker/librenms/custom.config.php:/opt/librenms/conf.d/custom.config.php:ro
–restart always
jarischaefer/docker-librenms:latest

Any further advice is welcome.

jasonloong · April 7, 2023, 10:43pm

Hi @Korvo

Thank you for the logs

Korvo:

[2023-04-07T20:07:35.154979+02:00] local.ERROR: PDOException: SQLSTATE[HY000] [1045] Access denied for user ‘librenms’@‘172.17.0.13’ (using password: YES) in /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:40
Stack trace:
[2023-04-07T22:37:47.122224+02:00] production.ERROR: PDOException: SQLSTATE[HY000] [1045] Access denied for user ‘librenms’@‘172.17.0.13’ (using password: YES) in /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:40

It seems like a permissions issue

Can you confirm that you have setup the sql user “librenms” with the correct permissions? (Replace libre_password with your_password)

CREATE DATABASE librenms CHARACTER SET utf8 COLLATE utf8_unicode_ci;
CREATE USER 'librenms'@'%' IDENTIFIED BY 'libre_password';
CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'libre_password';
GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'%';
FLUSH PRIVILEGES;
exit

Your MariaDB container volume mount has no persistent storage , volume is linked to the sql conf.d ~ unless you’ve setup the permissions above in the confd, the sql container starts from “fresh” whenever you restart the container

Korvo · April 8, 2023, 12:47am

I can confirm that I have put the passwords at ‘libre_password’
Yes you are right, this is my mistake and have changed it to:
-v /volume1/docker/mariadb:/var/lib/mysql \

However I got the same results when using the command:
docker exec librenms setup_database

CLI output:
Could not connect to database, check logs/librenms.log

I also noticed my PUID and PGID are not correct.
Command ‘ID’ shows this:

root@DiskStation:~# id
uid=0(root) gid=0(root) groups=0(root),2(daemon),19(log)

So I have to set it this way or am I mistaken?

-e PUID=0
-e PGID=0 \

Mabe also goot to mention I’m running DSM 7.1.1
Synology Docker version 20.10.3-1308

Any idea ?

jasonloong · April 8, 2023, 1:55am

Thanks for the reply @Korvo

Are the logs showing the same permission errors? with the new MariaDB container?

I would suggest just copy/paste the commands on my tutorial before changing passwords etc, just to test/make sure the commands work

start from beginning, remove both librenms and mariadb containers just in case they have any existing configs written when you attempted the tutorial
Note on MariaDB container setup, you will probably encounter 2002 error, wait for awhile, and run docker exec -it mariadb mysql -u root -p

docker stop librenms && docker rm librenms
docker stop mariadb && docker rm mariadb
cd /volume1/docker
rm -r mariadb
rm -r librenms

PUID/GUID

Do not have to set this, Synology runs Docker as root, just let the MariaDB container handle the permissions

Test run on my end

Everything works, i copy and paste the commands into my Syno as per the tutorial
Version: DSM 7.1.1-42962 Update 4

Let me know again if it fails~

Korvo · April 9, 2023, 7:59pm

Today I have copy and paste from your website and it works !!! THANKS !!!

However now I still don’t understand what went wrong in the first place

On thing I should mention about is the mariadb setting option that is required by LibreNMS: Installing LibreNMS - LibreNMS Docs

If I understand correctly they should be:

-e innodb_file_per_table=1 \
-e lower_case_table_names=0 \

instead of:

-- innodb_file_per_table=1 \
-- lower_case_table_names=0 \

Please correct me if I’m wrong.

jasonloong · April 10, 2023, 10:20am

Thanks for reporting in! Glad it works

I assume the first mistake was mounting the wrong volume, when MariaDB starts, it will “prepare” files, so somewhere, the permissions didn’t save, and when you issued the docker restart mariadb , the database did not have persistent storage, so it restarted into a “fresh” container (thus the permission errors, librenms does not exist)

Restart

If you still have time, you might want to revisit the copy/paste commands? Like change to your password, and timezone (you will need to clear the containers again though, as listed above)

The official docs saves the runtime config in a file, if we do -e in docker, it means it’s a environment variable in Linux. MariaDB docker do not read the environment variable above, so you would be starting mariadb without those arguments

Because it’s a docker container, they have a few environment variables you can set via the container, but not the above. You can see the EV they allow on the MariaDB docker page

Docker MariaDB

Because the Container EV does not include innodb_file_per_table , etc etc, the tutorial run them as arguments when MariaDB starts up in the container

MariaDB docker build tag @ latest

see last line of 23 CMD ["mariadbd"]

The double -- , are the arguments when starting MariaDB. The MariaDB container already runs mariadbd default, so putting these two arguments will result in mariadbd --argument1 --argument2

Tutorial should be easy

As a tutorial, I made things simple so that its easy to copy/paste, you can definitely make your own MariaDB config file, and then point the Docker container to read the config file for MariaDB container

docker run --name some-mariadb -v /my/custom:/etc/mysql/conf.d --rm mariadb:latest my_print_defaults --mysqld

The -v
-v /my/custom:/etc/mysql/conf.d

Inside /my/custom should be a my_custom_config.cnf, which should include the config for

# my_custom_config.cnf
# # check the proper config syntax to set these arguments
innodb_file_per_table=1
lower_case_table_names=0

so the tutorial docker run command will become:

docker run \
-v /volume1/docker/mariadb:/var/lib/mysql \ # the persistent storage
-v /my/custom:/etc/mysql/conf.d \ # the custom config folder
-e MYSQL_ROOT_PASSWORD='root_password' \
-e TZ=Asia/Singapore \
--name mariadb \
-d \
--restart always \
mariadb:latest

Hope that helps clear up some docker usage

Korvo · April 10, 2023, 7:41pm

Thank you Jason to clear these things up for me
Whish we both could grab a beer, soda or coffee together

Korvo · April 10, 2023, 8:16pm

And again I have a question

Why do we have to create two users in mariadb ?

CREATE DATABASE librenms CHARACTER SET utf8 COLLATE utf8_unicode_ci;
>>>> CREATE USER 'librenms'@'%' IDENTIFIED BY 'libre_password';
>>>> CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'libre_password';
GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'%';
FLUSH PRIVILEGES;
exit

I this not enough ?

CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'your-password'

jasonloong · April 10, 2023, 11:54pm

localhost in Docker context means only the container itself will have permissions to write to the DB

As the LibreNMS docs/installation expects you to install in a Linux instance, localhost makes sense as no one else should be able to access the DB from outside

Because we are using containers, you will need to add % so that any IP/network can have access to the container

For security hardening, you can set the container to only accept IP from another container for eg. 'librenms'@'172.17.0.3'

Korvo · April 12, 2023, 4:52pm

Thanks for the information.

It works now