Setting up NFS on Red Hat Ent. 64-bit Linux Boxes

Fri Jul 03 11:38:21 -0700 2009

Come August, we will need to be able handle a surge in traffic to our Quest Atlantis system. Load balancer with multiple app servers mounting an NFS volume is what we have in mind. The diagram below explains it pretty clearly hopefully:

From:

To:

As you can see, the goal that we hope to achieve from this new setup is to accommodate a much greater number of requests per second and to put us in a favorable position to scale horizontally quickly and easily.

This post is about setting up the NFS bit of the architecture. I also welcome you to consult the official NFS howto.

As per the objective diagram above, we will have one NFS server, two application servers that will mount a volume from the NFS server.

Setting up the NFS Server

Edit: /etc/exports
```
/dir/to/be/shared client_1_ip_address(rw,sync) client_2_ip_address(rw,sync)
```
Meaning, the ip address mentioned (with access conditions) are the only ones that will be allowed to mount the directory that is being shared on the NFS server. Note, It can either be IP Address or the domain name of the client machine. Keep in mind, IP address, domain name spoofing can be done to hack into the NFS volume and please consult the security section of the NFS documentation for further securing your setup.
Edit: /etc/hosts.deny
```
portmap: ALL
lockd: ALL
mountd: ALL
rquotad: ALL
statd: ALL
```
Meaning, anybody and everybody will NOT be allowed to connect to these services mentioned.

Edit: /etc/hosts.allow

portmap: client_1_ip_address, client_2_ip_address
lockd: client_1_ip_address, client_2_ip_address
mountd: client_1_ip_address, client_2_ip_address
rquotad: client_1_ip_address, client_2_ip_address
statd: client_1_ip_address, client_2_ip_address

Following up from point 2, you will allow only certain machines to connect the services offered up the NFS server.

Start the NFS Service as root

/sbin/service nfs start

/etc/init.d/nfs start

Verify that the NFS server / Should yield:

 program 100000 100000 100024 100024 100011 100011 100011 100011 100003 100003 100003 100021 100021 100021 100003 100003 100003 100021 100021 100021 100005 100005 100005 100005 100005 100005

The port numbers is up and running.

 class="regex">usr/sbin/rpcinfo -p 127.0.0.1 vers proto   port 2   tcp    111  portmapper 2   udp    111  portmapper 1   udp    927  status 1   tcp    930  status 1   udp   1016  rquotad 2   udp   1016  rquotad 1   tcp   1019  rquotad 2   tcp   1019  rquotad 2   udp   2049  nfs 3   udp   2049  nfs 4   udp   2049  nfs 1   udp  58662  nlockmgr 3   udp  58662  nlockmgr 4   udp  58662  nlockmgr 2   tcp   2049  nfs 3   tcp   2049  nfs 4   tcp   2049  nfs 1   tcp  35840  nlockmgr 3   tcp  35840  nlockmgr 4   tcp  35840  nlockmgr 1   udp    666  mountd 1   tcp    669  mountd 2   udp    666  mountd 2   tcp    669  mountd 3   udp    666  mountd 3   tcp    669  mountd can be different but the format should be identical. The port numbers you see here are what will need to get included into the firewall rule, so that the clients can successfully mount the the NFS volume.




File: /etc/sysconfig/iptables Add these lines before the COMMIT line. Please note the port numbers mentioned below correspond to the numbers you see in point 5 above. 
:STANDARD - [0:0]
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 111 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 111 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 927 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 927 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 930 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 930 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 1016 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 1016 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 1019 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 1019 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 2049 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 58662 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 58662 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 35840 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 35840 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 666 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 666 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 669 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 669 -j ACCEPT
 Then restart the firewall service as root for the rules to take effect. /sbin/service iptables restart




Make sure the NFS server box starts up the nfs and portmap services when and upon system reboot, start etc. In other words, on during run level 3, 4 and 5. 
/sbin/chkconfig --list portmap
portmap        	0:off	1:off	2:off	3:on	4:on	5:on	6:off
/sbin/chkconfig --list nfs
nfs            	0:off	1:off	2:on	3:on	4:on	5:on	6:off
 If you see any of 3, 4 or 5 are off, then run the following command as root /sbin/chkconfig nfs on
/sbin/chkconfig portmap on



That’s it! That’s about all it should take for you to setup the NFS server, so that app servers can mount the NFS volume.

Setting up the NFS Client

After the NFS server is up, it’s just stupid easy to mount the NFS volume on the client.



First check and see the the clients can see the NFS ports 
/usr/sbin/rpcinfo -p nfs_server_ip_address
 Note, it can either be an IP address or the qualified domain name. The result of which should be (bear in mind, the port numbers can vary, but the format should be identical): program vers proto   port
 100000    2   tcp    111  portmapper
 100000    2   udp    111  portmapper
 100024    1   udp    927  status
 100024    1   tcp    930  status
 100011    1   udp   1016  rquotad
 100011    2   udp   1016  rquotad
 100011    1   tcp   1019  rquotad
 100011    2   tcp   1019  rquotad
 100003    2   udp   2049  nfs
 100003    3   udp   2049  nfs
 100003    4   udp   2049  nfs
 100021    1   udp  58662  nlockmgr
 100021    3   udp  58662  nlockmgr
 100021    4   udp  58662  nlockmgr
 100003    2   tcp   2049  nfs
 100003    3   tcp   2049  nfs
 100003    4   tcp   2049  nfs
 100021    1   tcp  35840  nlockmgr
 100021    3   tcp  35840  nlockmgr
 100021    4   tcp  35840  nlockmgr
 100005    1   udp    666  mountd
 100005    1   tcp    669  mountd
 100005    2   udp    666  mountd
 100005    2   tcp    669  mountd
 100005    3   udp    666  mountd
 100005    3   tcp    669  mountd




Then on client 1 and 2, all you have to do is as root: 
mount nfs_server_ipaddress:/dir/that/is/shared /where/you/want/to/mount
 If you timeout or cannot resolve host errors, and you’re 100% sure that you followed all the steps properly, get in touch with me or the intarwebz forums to diagnose your issue.



To automatically mount the NFS volume upon system reboot, edit /etc/rc.d/rc.local with the following entry. 
/bin/mount nfs_server_ipaddress:/dir/that/is/shared /where/you/want/to/mount




That’s it! Have fun riding the NFS nirvana peeps. Cheers!




	
blog comments powered by Disqus

intarwebz traversals

Setting up NFS on Red Hat Ent. 64-bit Linux Boxes

From:

To:

Setting up the NFS Server

Setting up the NFS Client