Welcome to my blog
FreeBSD web stack
I was lately playing with ZFS on Linux. ZFS is getting increasingly easy on Linux, with Ubuntu Wily (15.10) having zfs packages in default repository. The last part is especially cool surprise. There are some bumps on CentOS 7 though, even though ZFS on Linux Project build packages regularly for this distribution. So far I understood, problem comes rather from CentOS 7 misshaps with modules building rather than from ZOL side.
But still, if you read my previous entry, you may have noticed, setting up ZFS on / is not a simple thing. I still did not get to a working state with Fedora. I think I'll try to play with CentOS in a VM soon, to get a feeling how far away are we from success - or how close. Since this is a must have for boot environments, which I am very fond of, I am watching progress on that front with real interest.
But there is another issue that irks me a bit, coming from illumos land. Observability. While there is a wealth of tracing frameworks for Linux out there, none so far is complete and finished. None so far is also as easy to learn and use as DTrace. When it comes to complete documentation, they all are far behind DTrace Guide also.
There is an operating system, which has pretty good driver coverage while being able to boot off ZFS, has beadm command implemented and a wealthy choice of system packages you'd expect from a server system: FreeBSD.
Honestly, I would prefer, myself, to go with a chosen illumos distribution. I think I may come up with similar post covering OpenIndiana or OmniOS soon. But for those of you, who don't feel comfortable entering lands of pretty exotic operating systems - here's very popular and well know one.
Installing packages, creating the pool
Let us consider a FreeBSD server with operating system installed on single SCSI drive with ZFS as a main filesystem. Let's also consider it have another SCSI drive for operating system pool mirroring and four SAS drives to host data on them.
Before we can start, lets prepare the freshly installed FreeBSD. My personal package manager of choice is pkg and text editor is vim. Issuing command:
root@~# pkg install vimwill do two things for me: install pkg command and follow with installing vim package.
There is one command I wish to show off. Since I've installed FreeBSD on ZFS filesystem, I can freely use beadm command:
root@:~ # pkg install beadm Updating FreeBSD repository catalogue... FreeBSD repository is up-to-date. All repositories are up-to-date. The following 1 package(s) will be affected (of 0 checked): New packages to be INSTALLED: beadm: 1.2.6 The process will require 31 KiB more space. 9 KiB to be downloaded. Proceed with this action? [y/N]: y Fetching beadm-1.2.6.txz: 100% 9 KiB 9.5kB/s 00:01 Checking integrity... done (0 conflicting) [1/1] Installing beadm-1.2.6... [1/1] Extracting beadm-1.2.6: 100% root@:~ # beadm create clean-install Created successfully root@:~ # beadm list BE Active Mountpoint Space Created default NR / 999.1M 2016-02-16 11:48 clean-install - - 148.0K 2016-02-16 12:08I hope one day it will be that easy for by favorite Linux distros too. :)
There is a very nice tool for FreeBSD to list block devices on the system: camcontrol. As a side note - I couldn't find a similar tool for Linux. Is there not a one or did I miss it?
root@:~ # camcontrol devlistFrom zpool status I know my system is installed on da0 and da1:
root@:~ # zpool status pool: zroot state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da0p3 ONLINE 0 0 0 da1p3 ONLINE 0 0 0 errors: No known data errorsThus drives we can use are da2 to da5. As a side note, da is a naming scheme for SCSI and SAS.
Lets create a storage pool, named datapool (my personal habit):
root@:~ # zpool create -f datapool mirror da2 da3 mirror da4 da5 root@:~ # zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT datapool 3.97G 62K 3.97G - 0% 0% 1.00x ONLINE - zroot 13.9G 478M 13.4G - 2% 3% 1.00x ONLINE - root@:~ # zpool status datapool pool: datapool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM datapool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 errors: No known data errors
I have create a separate filesystem, called data, within the datapool:
root@freebsdzfs:~ # zfs create datapool/data root@freebsdzfs:~ # zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 93.5K 3.84G 19K /datapool datapool/data 19K 3.84G 19K /datapool/data
I've installed postgresql 9.4 by running:
root@freebsdzfs:~ # pkg install postgresql94-server postgresql94-client postgresql94-contrib [...] echo 'postgresql_enable="YES"' >> /etc/rc.confI've omitted pkg's output for brevity. There are some lines on upgrading databases that I cut out. Last line makes sure postgres engine will start at system boot.
I've created a separate ZFS filesystem for the databases. This way I can set block size to be exact same as PostgreSQL's default one, 8 kilobytes, allowing for best performance.
root@freebsdzfs:~ # zfs create -o recordsize=8K datapool/postgres root@freebsdzfs:~ # zfs get all datapool/postgres NAME PROPERTY VALUE SOURCE [...] datapool/postgres recordsize 8K local [...]Again, other properties have been omitted for brevity.
You will now need to initialize the directory for use by PostgreSQL following their Online Documentation:
root@freebsdzfs:~ # chown -R pgsql:pgsql /datapool/postgres root@freebsdzfs:~ # su pgsql $ initdb -D /datapool/postgres/data The files belonging to this database system will be owned by user "pgsql". This user must also own the server process. The database cluster will be initialized with locale "C". The default database encoding has accordingly been set to "SQL_ASCII". The default text search configuration will be set to "english". Data page checksums are disabled. creating directory /datapool/postgres/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting dynamic shared memory implementation ... posix creating configuration files ... ok creating template1 database in /datapool/postgres/data/base/1 ... ok initializing pg_authid ... ok initializing dependencies ... ok creating system views ... ok loading system objects' descriptions ... ok creating collations ... ok creating conversions ... ok creating dictionaries ... ok setting privileges on built-in objects ... ok creating information schema ... ok loading PL/pgSQL server-side language ... ok vacuuming database template1 ... ok copying template1 to template0 ... ok copying template1 to postgres ... ok syncing data to disk ... ok WARNING: enabling "trust" authentication for local connections You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. Success. You can now start the database server using: postgres -D /datapool/postgres/data or pg_ctl -D /datapool/postgres/data -l logfile start $To tell postgres to start using those, you need to add another line to /etc/rc.conf file:
echo 'postgresql_data="/datapool/postgres/data"' >> /etc/rc.confYou should be now able to create databases and store them in /datapool/postgres directory, living on a mirrored ZFS storage.
Installing Apache HTTP Server
Following similar workflow as above, I've installed Apache web server and configured it to store its files within /datapool/www ZFS filesystem, that I have created specially for that purpose. I've also enabled compression for this dataset (lz4), since web pages should compress nicely, saving me both storage and physical read/write on hard drives.
root@freebsdzfs:~ # pkg install apache24-2.4.18 [...] root@freebsdzfs:~ # echo 'apache24_enable="yes"' >> /etc/rc.conf root@freebsdzfs:~ # zfs create -o compress=lz4 datapool/wwwDo not forget to change the Document Root path in Apache's configuration file:
root@freebsdzfs:~ # sed -ie 's;/usr/local/www/apache24/data;/datapool/www;' /usr/local/etc/apache24/httpd.confFinally, check the configuration and start the service:
root@freebsdzfs:~ # service apache24 configtest root@freebsdzfs:~ # service apache24 start
Just to have some additional fun, lets install a tomcat container to our FreeBSD server:
root@freebsdzfs:~ # pkg install tomcat8 [...]Remember about this step:
This OpenJDK implementation requires fdescfs(5) mounted on /dev/fd and procfs(5) mounted on /proc. If you have not done it yet, please do the following: mount -t fdescfs fdesc /dev/fd mount -t procfs proc /proc To make it permanent, you need the following lines in /etc/fstab: fdesc /dev/fd fdescfs rw 0 0 proc /proc procfs rw 0 0Start the tomcat server:
root@freebsdzfs:~ # cd /usr/local/apache-tomcat-8.0/bin/ root@freebsdzfs:/usr/local/apache-tomcat-8.0/bin # ./startup.sh Using CATALINA_BASE: /usr/local/apache-tomcat-8.0 Using CATALINA_HOME: /usr/local/apache-tomcat-8.0 Using CATALINA_TMPDIR: /usr/local/apache-tomcat-8.0/temp Using JRE_HOME: /usr/local Using CLASSPATH: /usr/local/apache-tomcat-8.0/bin/bootstrap.jar:/usr/local/apache-tomcat-8.0/bin/tomcat-juli.jar Tomcat started.
So why would I go the great lengths to move from known Ubuntu/CentOS/Arch/Debian/Gentoo/... environment to FreeBSD one? Because of mentioned previously DTrace goodness. The Linux tracing frameworks, and there seem to be many of them, are difficult to set up, when compared with illumos/FreeBSD DTrace. It just works.
First thing you may wish to do is follow link to Prefetch Technologies and enable DTrace probes for you Apache.