Creating SSL certificates for Apache

First we need to understand the concepts of how SSL works.

When a client app (browser) makes a connection to the server (Web Server) via SSL (normally port 443), the initial communication creates the encrypted channel. This process is done as follows:

1- Server provides a certificate file (signed by a CA) to the client.

2- Client verifies the certificate file with the root CA (Certificate Authority). For external services, we can use Verisign, and for internal services we can create our own CA server.

3- Client also verifies that the lookup Name matches the CN (Common Name) f the certificate.

4- Client accepts the connection or sends a warning if the requirements are not met.

5- Client starts to encrypt traffic using the Certificate file.

6- Server decrypts the data using the Private Key.

Basically, whoever has the certificate file can encrypt data but no decrypt. The only one that can decrypt the data is the private key, so make sure you store your private key on a safe place.

Getting Certificate Files

1- First generate the Private key

#openssl genrsa -out server.key 2048

2- Generate the request for a Certificate. This file must be sent to the CA ( Verisign or Internal CA) so it can be signed.

This request does not contain the private key but it will help the CA to signed it using their own private key.

# openssl req -new -key server.key -out server.csr

Then you will be ask several questions and the most important one is the Common Name. The Common Name must match the exact name to access the server that requests the certificate (CRT file).

When you buy certificates from Verisign they have a website where you post CSR, then they will send you a certificate file. For Microsoft CA, you will have to submit the CSR file as a base64 encoded PKCS #10, then Microsoft CA will return a CER file that will need to be converted to a certificate.

Microsoft CA conversion CER to CRT file.

# openssl x509 -in server.cer -inform d -out server.crt

Finally add to your apache configuration for SSL the following configuration:

SSLCertificateFile /path/to/ssl/certificate/server.crt

SSLCertificateKeyFile /path/to/ssl/key/server.key

Note: Make sure your key file are stored in a secure location.

Enjoy it !!!!

MySQL Cluster 7 Set up with Ubuntu

If you are accessing this page, it is possible that you want to figure out how difficult is to create a MySQL Cluster. I can tell you that the installation process is not that complicated, the most complicated part is to try to move current databases to this environment , since MySQL cluster engine (NDB) has some differences with other engines (MyISAM, InoDB, ..)

First, we need to understand that MySQL uses different storage engines. Each storage engine has their own advantage and disadvantage. However, NDB is the only engine that according to MySQL can guarantee HA.

Different MySQL configurations can guarantee some degree of HA. The most popular options are the following:

1- Master - Slave Replication : If your Master Server is down you can promote the Slave to master. This configuration is considered active - passive . Moreover, some data can be lost.

2- Master - Master Replication (Asynchronous) : This configuration is great for HA however you would have to have some considerations to avoid conflict. This configuration is considered active - active.

3- MySQL HA with DRBD: DRBD is a process that keep two server file system in sync. The synchronization process is at data block level, so database corruption can be seen. This configuration is active - passive.

4- MySQL Cluster: Basically, MySQL introduces a new storage engine that can help to maintain redundancy and HA.

MySQL Cluster

First, we need to recognize the different processes and facts for MySQL Cluster.

The NDB engine stores Indexes and Data on memory by default. However, if you define a tablespace on disk, Data can be stored on disk.

1- ndbd or ndbmt : Single or Multi thread process for data nodes. This process reserves the memory and stores the data on memory. (Data Nodes)

2- ndb_mgmd : This process provides the configuration file to all nodes (Data Nodes and MySQL Nodes).

3- Mysqld (API): This process provides all the storage engine available plus access to the NDB engine.

Starting the Set UP

Although you can run MySQL cluster in one machine , it does not make sense to install it since you do not have any redundancy. I suggest a basic setup of 5 nodes ( 2 data nodes, 2 mysql nodes and 1 management). Also the following specs will help you to get the HW.

1- Data Nodes:

- A lot of memory. Index and Data are mainly stored in memory.

- Disk space. If you store Data tables on disk.

- CPU. Data nodes are not CPU intensive. Moreover , if you use multiple cpu, you should use ndbmtd process.

2- MySQL Nodes:

- CPU. These Nodes are CPU intensive.

- Memory. As required.
3- Management Node.
- Any small machine can perform this task.

Moreover, it is suggested to have a different subnet for data communication between the data and sql nodes (security and traffic). According to this, sql nodes must have two interfaces, one for access from external apps and one for data communications to data nodes (NDB).

datanode01 192.168.0.1
datanode02 192.168.0.2

sqlnode01 192.168.0.3 172.1616.1
sqlnode02 192.168.0.4 172.16.16.2

mnode01 192.168.0.5

Installing Mysql Cluster.

You are probably tempted to install Mysql using apt-get , however the repos do not contain the latest version of Mysql, so I suggest you to use the binary installation of MySQL. Go to MySQL website and chose the Linux Generic Download, then arch 32 or 64 and finally the mirror.

# wget http://www.mysql.com/get/Downloads/MySQL-Cluster-7.0/mysql-cluster-gpl-7.0.13-linux-x86_64-glibc23.tar.gz/from/http://mysql.he.net/

# tar xzvf mysql-cluster-gpl-7.0.13-linux-x86_64-glibc23.tar.gz

Then
# cd mysql-cluster-gpl-7.0.13-linux-x86_64-glibc23

Read the INSTALL-BINARY file ...so you can install it.

shell> groupadd mysql
shell> useradd -g mysql mysql
shell> cd /usr/local
shell> gunzip < /path/to/mysql-VERSION-OS.tar.gz | tar xvf - shell> ln -s full-path-to-mysql-VERSION-OS mysql
shell> cd mysql
shell> chown -R mysql .
shell> chgrp -R mysql .
shell> scripts/mysql_install_db --user=mysql
shell> chown -R root .
shell> chown -R mysql data

Wait ...This is the basic installation for all the nodes and we do not want to have mysqld running on all the nodes, only sql nodes.

This command will be executed on sql nodes when all configuration files are ready:

shell> bin/mysqld_safe --user=mysql &


Setting Up Management Node.

Copy the template configuration file for management node.

#cp /usr/local/mysql/support-files/ndb-config-2-node.ini /usr/local/mysql/config.ini

This configuration contains pretty basic stuff that you would have to tweak for your cluster.
However, I can suggest you a couple of changes depending on the amount of memory that you have available.
Remember mysql cluster stores tables on memory, so if we have more memory available adjust the values.
For example, I have 32GB of memory on each data node.

DataMemory=25634M
IndexMemory=3205M

There are more changes that you can tweak so you can improve the behaviour of the cluster or you can try this website to help you configure the cluster. http://www.severalnines.com/config/

Modify the IP for the management node.

[ndb_mgmd] Id=1 HostName= 192.168.0.5


Then modify the IPs for data nodes under [ndbd]

[ndbd] Id= 2 HostName= 192.168.0.1 [ndbd] Id= 3 HostName= 192.168.0.2
Then the IPs for sql nodes under [mysqld]

[mysqld] Id= 4 HostName= 192.168.0.3 [mysqld] Id= 5 HostName= 192.168.0.4
Save the file.

Create the mysql-cluster directory.

# mkdir /var/lib/mysql-cluster

then Start the management node.

# /usr/local/mysql/bin/ndb_mgmd -f /usr/local/mysql/config.ini

If we wan to check the cluster use the following command.

# /usr/local/mysql/bin/ndb_mgm -- NDB Cluster -- Management Client -- ndb_mgm> show


Setting Up Data node

(I assume you have installed MySQL Binary)

Basically we need to tell the ndbd daemon to use configuration from management node.

Create configuration file on /etc/mysql/

#mkdir /etc/mysql
#vi /etc/mysql/my.cnf

[mysqld]
ndbcluster
# IP address of the cluster management node
ndb-connectstring=192.168.0.5

[MYSQL_CLUSTER]
ndb-connectstring=192.168.0.5

Because it is the first time we are going to start ndbd daemon we need to initialize it.

# /usr/local/mysql/bin/ndbd --initial

After both data nodes are up, you can check the status on the management node using the show command.

If you want to stop a data node , you can run the following command on the management node:

ndb_mgm> 2 stop

It will stop the datanode with ID=2 (see management node configuration)


Setting Up SQL Node or API.

(I assume you have installed MySQL Binary)

Create a my.cnf file.

#vi /etc/my.cnf

[mysqld]
ndbcluster
# IP address of the cluster management node
ndb-connectstring=192.168.0.5
[mysql_cluster]
# IP address of the cluster management node
ndb-connectstring=192.168.0.5

Finally start the SQL nodes.

# /usr/local/mysql/support-files/mysql.server start

After all the nodes are running you can monitor the nodes from the management nodes.

Note: Remember that you must create tables with ENGINE=NDB so they can be running on the cluster.

Enjoy it ...

Linux Hard Drive Performance measurements

Many people trust vendor specs, moreover they trust hard drive speed, disk controller speed and so on. However, reality is that you will never get those specs when you set up the hardware. Why? Vendors test their hardware on special environments, with specific combination of hardware.

I was trying to set up a server with fast access to hard drives so I did a little research of how to test different configuration of hard drives. These are my findings:

Note: All these tests should be executed several times. The real results are affected by hard drive cache, controller cache, and operative system cache.

1- (Read Speed) hdparm -t /dev/sda

This is a very well known command that It can give you the max read speed (sequential).

2- (Write Speed) dd count=1k bs=10M if=/dev/zero of=/data/test.img

This is another well known command that It can give you an average of writes.

This command creates a 10G file and measure the speed.

3- (Write speed) This people http://www.nlanr.net/Dskwtst/ created a C program that logs write speed.

#wget http://www.nlanr.net/Dskwtst/Software/dskwtst.c

#gcc -O2 dskwtst.c -o dskwtst

(Compiling probably you would get an error)

dskwtst.c: In function âmainâ:

dskwtst.c:32: warning: incompatible implicit declaration of built-in function âexitâ

This error is only a warning so it should not affect results.

#./dskwtst > /data/ouputfile 2> ./log

The command will create a file on /data/ and it will send the results to log.

4- (Random access)This guy http://www.linuxinsight.com/how_fast_is_your_disk.html created a C program that can measure the seek time of the hard drive.

#wget http://www.linuxinsight.com/files/seeker.c

#gcc -O2 seeker.c -o seeker

#./seeker /dev/sda

Enjoy it ...

Postfix relaying emails with GMAIL SMTP (Centos 5.3)

I was tired of maintaining my email server so I decide to move everything to Google app. They can host your mail server up to 50 accounts for free, isnt it great?. So I have moved all my domains to Google apps and everything was working perfect, however I missed an issue RELAY emails!!

So I relaxed and I figured out how to relay emails using an Google app (GMAIL) account.

After reading a lot of post, I realized that everybody was missing a part of the problem.

Some people shows you how to create client certificates for postfix when you do not need them.

Some people tell you to get the root CA certificates when you already have them.

However the most important part is to have all the required packages to make it work.

For Centos 5.3

Verify package installed

# rpm -qa |grep postfix

postfix-2.3.3-2.1.el5_2

# rpm -qa |grep sasl

cyrus-sasl-lib-2.1.22-5.el5

cyrus-sasl-2.1.22-5.el5

cyrus-sasl-plain-2.1.22-5.el5

# rpm -qa |grep openssl

openssl-perl-0.9.8e-12.el5_4.1

openssl-devel-0.9.8e-12.el5_4.1

xmlsec1-openssl-1.2.9-8.1.1

openssl-0.9.8e-12.el5_4.1

openssl097a-0.9.7a-9.el5_2.1

Copy root CA certificates. Postfix needs to know the location of root CA.

# cp /etc/pki/tls/certs/ca-bundle.crt /etc/postfix/cacert.pem

Create file that stores GMAIL user and password.

# vi /etc/postfix/sasl_passwd

smtp.gmail.com user@domain:password

#postmap /etc/postfix/sasl_passwd

Edit /etc/postfix/main.cf

# Relay all e-mail via GMail.

relayhost = [smtp.gmail.com]:587

# SASL authentication

smtp_sasl_auth_enable=yes

smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

smtp_sasl_security_options =

smtp_sasl_tls_security_options = noanonymous

smtp_sasl_mechanism_filter = login

# TLS

smtp_tls_eccert_file =

smtp_tls_eckey_file =

smtp_use_tls = yes

smtp_enforce_tls = no

smtp_tls_CAfile = /etc/postfix/cacert.pem

smtpd_tls_received_header = yes

tls_random_source = dev:/dev/urandom

Finally, restart postfix.

Enjoy it ...

Installing Ubuntu from USB drive

I am not a very big Fan of Ubuntu, however Ubuntu has a lot of acceptance in the Opensource Community. So I decided that I would have to give a try.

On this Blog, I will present an installation method that it is very helpful for many System Admin.

Let's review the installation methods first:

1- Installing from CD. You will have to download the ISO image from Ubuntu, have a ISO burner software, a CD , and a CD drive. Today, many servers do not bring CD unit since they are a waste of space and hardware.

When you Installing from CD, you can have two options:

- Installing all packages from CD

- Installing all packages from Repositories. (You need DHCP, Repository, and network enable)

2- Installing from Network. This method is great however Ubuntu is not very good at this. You will need DHCP, TFTP, and Repository. Moreover, you will need to write down the MAC address of the server so you can configure DHCP and create a autoconfiguration file.

3- Installing from USB. This method is a variation of the CD, however Ubuntu is not straight forward with this installation method. There are many tutorials and blogs about installing Ubuntu and other Linux flavors, using USB drives, however they do not specify the complications during the installation process. To make an USB drive bootable is a very simple process, however making the installation from USB for Ubuntu is not.

My problem: I have a HP server with RAID controller (1 Logical Volume), no network connection, and no CD drive. I need to install Ubuntu 9.10 (Although it works for other versions too). I have my Laptop running with Windows (Creating a USB installer on Linux is a simpler process)

1- First get syslinux for Windows.

http://reilly.homeip.net/folding/usb.html

2- Download Ubuntu 9.10

3- Open ubuntu.iso file and copy isolinux dir from ISO to / USB.

There are many options to do that. You can mount the ISO as Virtual CD using http://poweriso.com/ or you can use an Zip software to grab that directory.

4- Rename directory isolinux to syslinux .

On isolinux dir, you can find isolinux.cfg, rename it to syslinux.cfg.

5- Create a directory named install on / USB drive.

6- Get vmlinuz and initrd.gz files for hd-media.

http://archive.ubuntu.com/ubuntu/dists/karmic/main/installer-amd64/current/images/hd-media/

7- Copy vmlinuz and initrd.gz files to /install USB drive.

8- Run syslinux.exe -m Drive: (USB Drive letter)

9- Copy ISO image for Karmic-amd64 on USB

10- Boot from USB ( I assume that you know how to set up the BIOS to boot from USB)

11- Follow the installation process.

12- Before the end, when the installation has ended, you will have to make some modifications to grub. If you miss this part the server will not boot since the grub is looking for the USB Drive.

Press alt-F2.

On The command line:

# chroot /target

/target is the mounted root partition.

#vi /boot/grub/device.map

Delete the entry for (hd0) that set to USB

Modify (hd1) to (hd0) on your boot drive

Finally, set grub on the hard drive.

# grub-install

#exit

Then Boot server.

All done.