Setting up NFS on Red Hat Ent. 64-bit Linux Boxes

nfs sysadmin linux

Fri Jul 03 11:38:21 -0700 2009

Come August, we will need to be able handle a surge in traffic to our Quest Atlantis system. Load balancer with multiple app servers mounting an NFS volume is what we have in mind. The diagram below explains it pretty clearly hopefully:

From:

quest atlantis old system architecture

To:

quest atlantis new system architecture
As you can see, the goal that we hope to achieve from this new setup is to accommodate a much greater number of requests per second and to put us in a favorable position to scale horizontally quickly and easily. This post is about setting up the NFS bit of the architecture. I also welcome you to consult the official NFS howto. As per the objective diagram above, we will have one NFS server, two application servers that will mount a volume from the NFS server. #Setting up the NFS Server# 1. Edit: **/etc/exports** 
/dir/to/be/shared client_1_ip_address(rw,sync) client_2_ip_address(rw,sync)
Meaning, the ip address mentioned (with access conditions) are the only ones that will be allowed to mount the directory that is being shared on the NFS server. Note, It can either be IP Address or the domain name of the client machine. Keep in mind, IP address, domain name spoofing can be done to hack into the NFS volume and please consult the security section of the NFS documentation for further securing your setup. 2. Edit: **/etc/hosts.deny** 
portmap: ALL
lockd: ALL
mountd: ALL
rquotad: ALL
statd: ALL
Meaning, anybody and everybody will NOT be allowed to connect to these services mentioned. 3. Edit: **/etc/hosts.allow** 
portmap: client_1_ip_address, client_2_ip_address
lockd: client_1_ip_address, client_2_ip_address
mountd: client_1_ip_address, client_2_ip_address
rquotad: client_1_ip_address, client_2_ip_address
statd: client_1_ip_address, client_2_ip_address
Following up from point 2, you will allow only certain machines to connect the services offered up the NFS server. 4. Start the NFS Service as root 
/sbin/service nfs start
OR 
/etc/init.d/nfs start
5. Verify that the NFS server is up and running. 
   /usr/sbin/rpcinfo -p 127.0.0.1
Should yield: 
   program vers proto   port
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp    927  status
    100024    1   tcp    930  status
    100011    1   udp   1016  rquotad
    100011    2   udp   1016  rquotad
    100011    1   tcp   1019  rquotad
    100011    2   tcp   1019  rquotad
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100003    4   udp   2049  nfs
    100021    1   udp  58662  nlockmgr
    100021    3   udp  58662  nlockmgr
    100021    4   udp  58662  nlockmgr
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100021    1   tcp  35840  nlockmgr
    100021    3   tcp  35840  nlockmgr
    100021    4   tcp  35840  nlockmgr
    100005    1   udp    666  mountd
    100005    1   tcp    669  mountd
    100005    2   udp    666  mountd
    100005    2   tcp    669  mountd
    100005    3   udp    666  mountd
    100005    3   tcp    669  mountd
The port numbers can be different but the format should be identical. The port numbers you see here are what will need to get included into the firewall rule, so that the clients can successfully mount the the NFS volume. 6. File: **/etc/sysconfig/iptables** Add these lines *before* the COMMIT line. Please note the port numbers mentioned below correspond to the numbers you see in point 5 above. 
:STANDARD - [0:0]
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 111 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 111 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 927 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 927 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 930 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 930 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 1016 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 1016 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 1019 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 1019 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 2049 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 58662 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 58662 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 35840 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 35840 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 666 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 666 -j ACCEPT
-A STANDARD -m state --state NEW -m tcp -p tcp --dport 669 -j ACCEPT
-A STANDARD -m state --state NEW -m udp -p udp --dport 669 -j ACCEPT
Then restart the firewall service as root for the rules to take effect. 
/sbin/service iptables restart
7. Make sure the NFS server box starts up the nfs and portmap services when and upon system reboot, start etc. In other words, on during run level 3, 4 and 5. 
/sbin/chkconfig --list portmap
portmap                0:off        1:off        2:off        3:on        4:on        5:on        6:off
/sbin/chkconfig --list nfs
nfs                    0:off        1:off        2:on        3:on        4:on        5:on        6:off
If you see any of 3, 4 or 5 are off, then run the following command as root 
/sbin/chkconfig nfs on
/sbin/chkconfig portmap on
That's it! That's about all it should take for you to setup the NFS server, so that app servers can mount the NFS volume. #Setting up the NFS Client# After the NFS server is up, it's just stupid easy to mount the NFS volume on the client. 1. First check and see the the clients can see the NFS ports 
/usr/sbin/rpcinfo -p nfs_server_ip_address
Note, it can either be an IP address or the qualified domain name. The result of which should be (bear in mind, the port numbers can vary, but the format should be identical): 
   program vers proto   port
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp    927  status
    100024    1   tcp    930  status
    100011    1   udp   1016  rquotad
    100011    2   udp   1016  rquotad
    100011    1   tcp   1019  rquotad
    100011    2   tcp   1019  rquotad
    100003    2   udp   2049  nfs
    100003    3   udp   2049  nfs
    100003    4   udp   2049  nfs
    100021    1   udp  58662  nlockmgr
    100021    3   udp  58662  nlockmgr
    100021    4   udp  58662  nlockmgr
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100021    1   tcp  35840  nlockmgr
    100021    3   tcp  35840  nlockmgr
    100021    4   tcp  35840  nlockmgr
    100005    1   udp    666  mountd
    100005    1   tcp    669  mountd
    100005    2   udp    666  mountd
    100005    2   tcp    669  mountd
    100005    3   udp    666  mountd
    100005    3   tcp    669  mountd
2. Then on client 1 and 2, all you have to do is as root: 
mount nfs_server_ipaddress:/dir/that/is/shared /where/you/want/to/mount
If you timeout or cannot resolve host errors, and you're 100% sure that you followed all the steps properly, get in touch with me or the intarwebz forums to diagnose your issue. 3. To automatically mount the NFS volume upon system reboot, edit **/etc/rc.d/rc.local** with the following entry. 
/bin/mount nfs_server_ipaddress:/dir/that/is/shared /where/you/want/to/mount
That's it! Have fun riding the NFS nirvana peeps. Cheers!
blog comments powered by Disqus