| Menu principal |
| Logins |
FreeBSD Jails |
Afficher la révision 38How to communicate between FreeBSD? JailsTL;DR This article is about contructing two jails : a jail, named dbase, with MySQL?, the other, named www, with nginx. Jail www is visible on internet and displays table contents from a dabase on jail dbase. The purpose is to install every thing the hard way to undersstand hos to communicate between the two jails.
+Create the jailsThe install is made on a server named zoulou, IP 192.168.1.101, ethernet bge0, with FreeBSD?-12.2-Release. The jails are configured with ZFS on a local HD previously zfs'd zroot: root# zfs create zroot/jails root# zfs create zroot/jails/www root# zfs create zroot/jails/dbase root# mkdir /usr/local/jails/.dist-files Now I populate the ./dist-files directory with the base.txz file which I will use for the creation of the jails. For this install, I only need the base.txz package.
root# fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/12.2-RELEASE/base.txz -o /usr/local/jails/.dist-files root# tar xvf /usr/local/jails/.dist-files/base.txz -C /usr/local/jails/www root# tar xvf /usr/local/jails/.dist-files/base.txz -C /usr/local/jails/dbase Of course, there are other ways to do this; you can use zfs snaphosts/zfs clone. YMMV To update the FreeBSd? version of the jails and verify the checksums (IDS line), I must do this: root# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/www fetch install root# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/www IDS root# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/dbase fetch install root# env UNAME_r=12.2-RELEASE freebsd-update -b /usr/local/jails/dbase IDSAs the files in the jails has not yet been modified, there should be no errors with the checksum thing.
+Configuring the jails with a shared directoryThere are three main ways to allow jails to communicate:
The share directory is a way to share a typical info: the famous mysql.sock file, which is the unix socket to access the MySQL? (or MariaDB?) databases. It's rather simple: a directory is created on the host and on every jail; when the jails starts, the host directory is nullfs mounted on every jail. This happens in /etc/jail.conf 1- First, I create the directories: (feel free to choose any name or place for these directories.) root# mkdir /usr/local/jails/usocket root# mkdir /usr/local/jails/www/usocket root# mkdir /usr/local/jails/dbase/usocket2- Then, on the host, I must provide a way to mount /usr/local/jails/usocket every time a jail starts. I create two file named /etc/fstab.www and /etc/fstab.dbase.
+code for /etc/fstab.*# <2021-01-26 18:22:43 cmic> to mount the socket file of mysql. # fstab.www # Device Mountpoint FStype Options Dump Pass# /usr/local/jails/usocket /usr/local/jails/www/usocket nullfs rw,late 0 0 # #END # <2021-01-26 18:22:43 cmic> to mount the socket file of mysql. # fstab.dbase # Device Mountpoint FStype Options Dump Pass# /usr/local/jails/usocket /usr/local/jails/dbase/usocket nullfs rw,late 0 0 # #END And now for the network. I would like to config every jail with a valid IP address, so I could see/ping them on the LAN. To do that, I need a bridge and ethernet epairs to configure IP adresses to these two jails. FIXME vnet ? The usual way to this is ifconfig epair create which yields the ethernet pseudo interfaces named e0a and e0b. This where the commands jib and jng are useful. jib is used in /etc/jail.conf (will see this later) to manage vnet interfaces and link them to a bridge. But to use them, I first copy these commands so they could be found in the PATH (provided /usr/local/bin is in your PATH!) :
root# cp/usr/share/examples/jails/{jib,jng} /usr/local/sbin
So here is the /etc/jail.conf of the host zoulou:
+/etc/jail.conf
# <2021-01-21 14:40:39 cmic>
# adapted from the following script:
# https://www.cyberciti.biz/faq/configuring-freebsd-12-vnet-jail-using-bridgeepair-zfs/
#
# Todo first:
# cp /usr/share/examples/jails/jib /usr/local/sbin
# cp /usr/share/examples/jails/jng /usr//local/sbin
# chmod +x /usr/sbin/jib /usr/local/sbin/jng
#
# -hard-coded global variables
# -ifconfig the jail and defaultroute
#
$default_route="192.168.1.254";
$host_eth="bge0";
$ifconfig="/sbin/ifconfig";
$route="/sbin/route";
$jib="/usr/local/sbin/jib";
$jng="/usr/local/sbin/jng";
# To share a dir for mysql.sock
mount.fstab = "/etc/fstab.${name}";
#
# The www jail. With nginx (and PHP or Perl?)
#
www {
$jail_ip="192.168.1.202";
host.hostname = ${name}; # hostname
path = "/usr/local/jails/${name}"; # root directory
exec.clean;
exec.system_user = "root";
exec.jail_user = "root";
#####
vnet;
vnet.interface = "e0b_${name}"; # vnet interface(s)
exec.prestart += "${jib} addm ${name} ${host_eth}";
exec.poststop += "${jib} destroy ${name}";
# Standard stuff
exec.start += "/bin/sh /etc/rc";
exec.start += "${ifconfig} e0b_${name} inet ${jail_ip} up";
exec.start += "${route} add default ${default_route}";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_${name}_console.log";
mount.devfs; #mount devfs
allow.raw_sockets; #allow ping-pong
devfs_ruleset="5"; #devfs ruleset for this jail
allow.set_hostname = 1;
}
#
# The MySQL jail
#
dbase {
$jail_ip="192.168.1.203";
host.hostname = ${name}; # hostname
path = "/usr/local/jails/${name}"; # root directory
exec.clean;
exec.system_user = "root";
exec.jail_user = "root";
#####
vnet;
vnet.interface = "e0b_${name}"; # vnet interface(s)
exec.prestart += "jib addm ${name} ${host_eth}";
exec.poststop += "jib destroy ${name}";
# Standard stuff
exec.start += "/bin/sh /etc/rc";
exec.start += "${ifconfig} e0b_${name} inet ${jail_ip} up";
exec.start += "${route} add default ${default_route}";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_${name}_console.log";
mount.devfs; #mount devfs
allow.raw_sockets; #allow ping-pong
devfs_ruleset="5"; #devfs ruleset for this jail
allow.set_hostname = 1;
}
#
#END
Some explanations of jail.conf:
I have to configure /etc/rc.conf, localtime, /etc/resolv.conf, /etc/hosts in each jail. Here is a rc.conf.sample that I copy in /etc in each jail and modify if necessary:
+rc.conf.sample# # Example of rc.conf for a jail. # Customize this file ! # sendmail_enable="NONE" keymap="fr.iso.acc" #keymap="fr" defaultrouter="192.168.1.254" sshd_enable="YES" zfs_enable="YES" ###### And I cp this files too: root# cp /etc/hosts /usr/local/jails/www/etc root# cp /etc/localtime /usr/local/jails/www/etc root# cp /etc/resolv.conf /usr/local/jails/www/etcI enable the jails on the host, start the jails, and list them with jls: Notice that the IP adress field is empty, which is not a problem.
root# sysrc jail_enable="YES"
root# service jail start
Starting jails: www dbase.
root# jls
JID IP Address Hostname Path
7 www /usr/local/jails/www
8 dbase /usr/local/jails/dbase
root#
To verify, I ping a jail. I can ping a jail from the host or from anywhere in the 192.168.1.0/24 in the LAN.
root# ping -c 1 192.168.1.203
And now, for info, here the result of the command ifconfig on the host, where I can see: the bge0bridge with attached interfaces: bge0 (native eth interface), e0a_www and e0a_dbase. All this was done by the jib command in jail.conf
+ifconfig on the host
bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c0099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
ether 2c:41:38:87:3e:d4
inet 192.168.1.101 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bge0bridge: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:11:c2:f2:6e:00
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: e0a_dbase flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000
member: e0a_www flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000
member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 2 priority 128 path cost 20000
groups: bridge
nd6 options=1<PERFORMNUD>
e0a_www: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:40:d0:87:3e:d4
hwaddr 02:71:dc:ce:e1:0a
inet6 fe80::71:dcff:fece:e10a%e0a_www prefixlen 64 scopeid 0x5
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
e0a_dbase: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:40:c9:87:3e:d4
hwaddr 02:64:69:56:1c:0a
inet6 fe80::64:69ff:fe56:1c0a%e0a_dbase prefixlen 64 scopeid 0x6
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Now that the jails are running, I add hostname variable on each jail: root# sysrc -j www hostname="www" hostname: -> www root# sysrc -j dbase hostname="dbase" hostname: -> dbase I "log" on a jail with the command jexec and, for info, I look at the result of the ifconfig commend:
root# jexec www
root@www:/ #ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
e0b_www: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 0e:40:d0:87:3e:d4
hwaddr 02:71:dc:ce:e1:0b
inet 192.168.1.202 netmask 0xffffff00 broadcast 192.168.1.255
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root#
I can see the other end of epair, e0b_www and its IP address. So I verify that I can ping any host on the LAN. Everything OK. Now I have to install more software on the jail.
+Installing software in the jailsAlways logged in the jail ? If not, I do a jexec www. Then I install nginx, enable it, and launch it.
root@www:/ #pkg search nginx ... nginx-1.18.0_45,2 ... root# pkg install nginx-1.18.0_45,2 ... The following 2 package(s) will be affected (of 0 checked): New packages to be INSTALLED: nginx: 1.18.0_25,2 pcre: 8.44 ... root# sysrc nginx_enable="YES" nginx_enable -> YES root# service nginx start Performing sanity check on nginx configuration: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful Starting nginx. root@www:/ # I verifiy that nginx works OK by starting a browser on a host in the LAN, address pointing to 192.168.1.202 (www jail IP address). I should see a "Welcome to nginx" message.. I install PHP to this jail and the module fpm needed for nginx, but it is a bit out of topic. There are many tutos for this on internet. Here are 2 links:
I install Perl (pkg install perl5-5.32.0_1), instead, with p5-DBD-mysql-4.050 and p5-DBI-1.643 Due to security problems on my LAN, I configure nginx to listen on port 8080 (instead of port 80). I also modify the configuration file of nginx according to this guide https://kifarunix.com/install-nginx-mysql-php-femp-stack-on-freebsd-12/
root@www:/ # vi /usr/local/etc/nginx/nginx.conf
...
modify lines like this:
user www;
worker-processes 2;
error_log /var/log/nginx/error.log info;
access_log /var/log/nginx/access.log;
server_namme: www;
in the section '''server {'''
modify the line:
listen 80;
to read:
listen 8080;
in the section '''server {''''''
modify the line:
index index.html index.htm;
to read:
index index.php index.html index.htm;
root@www:/ # service nginx restart
Back to the host to install MySQL? on the dbase jail.
root@www:/ #exit root# jexec dbase root@dbase:/ # pkg search mysql .. mysql80-client-8.0.22_1 Multithreaded SQL database (client) mysql80-server-8.0.22_1 Multithreaded SQL database (server) .. root@dbase:/ # pkg install mysql80-server-8.0.22_1 ... Number of packages to be installed: 21 The process will require 457 MiB more space. 57 MiB to be downloaded. Proceed with this action? [y/N]:y ... root@dbase:/ #Notice that MySQL? client is and Perl are also installed! I used info from this link to install MySQL? correctly: https://kifarunix.com/install-nginx-mysql-php-femp-stack-on-freebsd-12/. In my case, I also want to use a unix domain socket on a decicated directory nullfs mounted (remember ?). So I have to modify the file my.cnf to point to the right directory:
root@dbase:/ # vi /usr/local/etc/mysql/my.cnf In the [client] section, modify the socket line to: socket = /usocket/mysql.sockIn the [server] section, modify the socket line to: socket = /usocket/mysql.sock
root@dbase:/ # sysrc mysql_enable="YES" mysql_enable: -> YES root@dbase:/ # service mysql-server start Starting mysql. I verify that the famous unix socket is in the right place:
root@dbase:/ #ls -l /usocket Huh? no unix socket here ? I must verify somthing:
Classic tip for permissions (88 is the userid and groupid of mysql):
root# service jail start root# jexec dbase root@dbase:/ # chown mysql /usocket && chgrp mysql /usocket root@dbase:/ # exit root# service jail restart .. root#jexec www root@www:/ # ls -l /usocket/ total 1 srwxrwxrwx 1 88 88 0 Feb 5 22:26 mysql.sock -rw------- 1 88 88 6 Feb 5 22:26 mysql.sock.lock root@www:/ # I can also verify the presence of the unix socket with the command netstat:
root@dbase:/ #netstat -an ... Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr fffff80010225b00 stream 0 0 fffff8004a0d5b40 0 0 0 /usocket/mysql.sock fffff80010225c00 stream 0 0 fffff8003d6e45a0 0 0 0 /tmp/mysqlx.sock ... root@dbase:/ # MySQL? server is running OK. Now I add a user cmic1 who can connect from anywhere on the lan, whith a password MyPass?$2. Then I create a database with one table and some dummy data. The purpose is to have a useful set of data to work with. The full list of instructions is here:
+Creating data in MySQL?
root@dbase:/ #mysql root@localhost [(none)]>CREATE USER 'cmic1'@'%' IDENTIFIED BY 'MyPass?$2'; Query OK, 0 rows affected (0.03 sec) root@localhost [(none)]>SELECT user,authentication_string,plugin,host FROM mysql.user; +------------------+------------------------------------------------------------------------+-----------------------+-----------+ | user | authentication_string | plugin | host | +------------------+------------------------------------------------------------------------+-----------------------+-----------+ | cmic1 | $A$005$7|@y(m(E:>0^e@rv,%tQv3zIxEMizVb3gvyAJunrBv9Ocf9/DRKL4QtwNOyz4 | caching_sha2_password | % | | mysql.infoschema | $A$005$THISISACOMBINATIONOFINVALIDSALTANDPASSWORDTHATMUSTNEVERBRBEUSED | caching_sha2_password | localhost | | mysql.session | $A$005$THISISACOMBINATIONOFINVALIDSALTANDPASSWORDTHATMUSTNEVERBRBEUSED | caching_sha2_password | localhost | | mysql.sys | $A$005$THISISACOMBINATIONOFINVALIDSALTANDPASSWORDTHATMUSTNEVERBRBEUSED | caching_sha2_password | localhost | | root | | caching_sha2_password | localhost | +------------------+------------------------------------------------------------------------+-----------------------+-----------+ 5 rows in set (0.00 sec) root@localhost [(none)]>create database mydb; Query OK, 1 row affected (0.03 sec) root@localhost [(none)]>use mydb; root@localhost [mydb]> CREATE TABLE mytable -> ( -> id int unsigned NOT NULL auto_increment, -> username varchar(100) NOT NULL, -> email varchar(100) NOT NULL, -> PRIMARY KEY (id) -> ); Query OK, 0 rows affected (0.15 sec) OK. A database with one table is created. Now I must insert some data in it, then look at the table content:
root@localhost [mydb]>insert into mytable (username, email) values ("cmic", "cmic@bigfoot.com");
Query OK, 1 row affected (0.03 sec)
root@localhost [mydb]>insert into mytable (username, email) values ("gaston", "king-bee@hive.net");
Query OK, 1 row affected (0.03 sec)
root@localhost [mydb]>
root@localhost [mydb]>
root@localhost [mydb]>
root@localhost [mydb]>select id, username, email form mytable;
+----+----------+------------------------+
| id | username | email |
+----+----------+------------------------+
| 1 | cmic | cmic@bigfoot.com |
| 2 | gaston | king-bee@hive.net |
+----+----------+------------------------+
2 rows in set (0.00 sec)
And last but not least, grant access to smic to mydb database:
root@localhost [mydb]>GRANT SELECT, INSERT, UPDATE ON databaseName.* TO 'cmic1'@'%'; Query OK, 0 rows affected (0.02 sec) root@localhost [mydb]>flush privileges; Query OK, 0 rows affected (0.02 sec) root@localhost [mydb]>exit; Bye root@dbase:/ # Warning: I am not a Mysql guru, so there might be errors in this database installment. Forgive me. I only want to make a simple example that works.
root@dbase:/ # exit; Bye root@dbase:/ # I am now ready to get the info of mydb from the www jail.
+MySQL? is in another jail !We go in jail www and prepare a Perl cript to access the database mydb thru the unix socket on /usocket. The Perl script on www jail:
+/testdb.pl
#!/usr/local/bin/perl
#
# Testing a databse on jail dbase, Ip 192.168.1.203
#
use strict;
use warnings;
use v5.10; # for say() function
#
use DBI;
say "Perl MySQL Connect Demo";
say "-----------------------";
#
# database name is mydb
# How to connect to database on another host/jail??
my $database="mydb";
my $portnum="3306";
my $dsn = "DBI:mysql:database=$database;mysql_socket=/usocket/mysql.sock;port=$portnum";
my $username = "cmic1";
my $password = 'MyPass$2';
my @res;
my $dbh = DBI->connect($dsn,$username,$password) or
die("Error connecting to the database: $DBI::errstr\n");
say "Connected to MySQL on jail dbase, mydb database \n";
#
# extract data
#
my $prepare=$dbh->prepare('select id, username, email from mytable') or die $dbh->errstr;
$prepare->execute() or die "Echec requete\n";
#
my ($id, $name, $mail);
while (($id, $name, $mail)=$prepare->fetchrow()) {
print $id . " " . $name . " " . $mail . "\n";
}
#
#END
And I execute the script :
root@www:/ # chmod +x testdb.pl root@www:/ # ./testdb.pl Perl MySQL? Connect Demo -------------------------- Connected to MySQL? on jail dbase, mydb database
1 cmic cmic@bigfoot.com 2 gaston king-bee@hive.net root@www:/ # OK. It works as expected. Now I have a php script /usr/local/www/nginx/index.php on jail www:
+index.php on jail www
<?php error_reporting(0); // $dbname = 'mydb'; $dbuser = 'cmic1'; $dbpass = 'MyPass?$2'; $dbhost = 'localhost'; // $link = new mysqli('localhost', $dbuser, $dbpass, "", 3306, '/usocket/mysql.sock'); if ($link->connect_errno) { echo "Echec lors de la connexion MySQL? : (" . $link->connect_errno . ") " . $link->connect_error; echo "\n"; } // some info echo $link->host_info . " (via host_info) \n"; mysqli_select_db($link, $dbname) or die("Could not open the db '$dbname'"); $sql="select id, username, email from mytable"; $result=$link->query($sql); // we get the data !! if ($result->num_rows > 0) { // output data of each row while($row = $result->fetch_assoc()) { echo "Id: " . $row["id"]. " > Name: " . $row["username"]. " > Email: " . $row["email"] . "\n"; } } else { echo "0 results"; } $link->close(); ?>
root@www:/ # php index.php Echec lors de la connexion MySQL? : (2054) The server requested authentication method unknown to the client. root@www:/ # Argh! Access denied. This one is stupid. I found out that the culprit is the default encryption method for password with MySQL? version 8. Now it is caching_sha2_password, but php doesn't understand this method for now. There is a fix to this: we go back to old encryptin methos in MySQL?:
root # jexec dbase root@dbase:/ #cd /usr/local/etc/mysql root@web:/usr/local/etc/mysql #vi my.cnf In section [mysqld], add the followin line:
default_authentication_plugin = mysql_native_password root@web:/usr/local/etc/mysql #cd / root@dbase:/ # service mysql-server restart Stopping mysql. Waiting for PIDS: 41024. Starting mysql. root@dbase:/ # mysql root@localhost [(none)]>alter user 'cmic1'@'%' identified with mysql_native_password by 'MyPass?$2'; root@localhost [(none)]>select user, authentication_string, plugin from mysql.user; +------------------+------------------------------------------------------------------------+-----------------------+ | user | authentication_string | plugin +------------------+------------------------------------------------------------------------+-----------------------+ | cmic1 | *9824B3D032944E5667EB0D380094F9B02CC39A2A | mysql_native_password | mysql.infoschema | $A$005$THISISACOMBINATIONOFINVALIDSALTANDPASSWORDTHATMUSTNEVERBRBEUSED | caching_sha2_password ... root@localhost [(none)]>exit; root@dbase:/ # exit root # jexec www root@www:/ # php /usr/local/www/nginx/index.php Localhost via UNIX socket (via host_info) Id: 1 > Name: cmic > Email: cmic@bigfoot.com Id: 2 > Name: gaston > Email: king_beel@hive.net
+Testing the whole stuf
lll
|
