-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Ed, what I did a long time for similar reasons might help. I created a small script in prepre-start.d that mounted a tmpfs on /dev and created the necessary nodes to init the system. Did fine :) Oliver Edward Capriolo schrieb: > I have found an issue. > > It seems that glusterfs and or fuse is forcing a nodev option on my mount point. > > cat /etc/mtab > glusterfs /opt/jtg/mnt/shared fuse > rw,nosuid,nodev,allow_other,default_permissions,max_read=1048576 0 0 > > So /dev can not be on gluster. > I have a way to get 99% of the system files onto gluster. This way > works but i feel like I missed something obvious. > > [root@jtg111 ~]# cat /etc/vservers/test01/fstab > none /proc proc defaults 0 0 > none /tmp tmpfs size=16m,mode=1777 0 0 > none /dev/pts devpts gid=5,mode=620 0 0 > /opt/jtg/mnt/shared/vservers/glustertest/ /data none bind > /opt/jtg/mnt/shared/vservers/glustertest/bin /bin node bind > /opt/jtg/mnt/shared/vservers/glustertest/usr /usr node bind > /opt/jtg/mnt/shared/vservers/glustertest/var /var node bind > .... > This way works. It has two obvious flaws. 1 if someone creates a > directory off of '/' like '/opt', it will be on the local disk, not > the gluster disk. It is a pain to manage one entry per file system off > the root. > > So the questions. > 1) Can I get gluster to store and serve a device file. ...If so is > that really my vserver problem or am i hunting in the wrong direction? > 2) Is there a way to configure fstab to mount / with the exception of > a directory like /dev, that would still have to be local? > > > On Wed, Sep 10, 2008 at 4:07 PM, Edward Capriolo <edlinuxguru@gmail.com> wrote: >> Ok! the day has finally come that I have two nodes for testing! My >> Goal: share an active/passive guest across two nodes with glusterfs. >> So here is what I have so far.....My first attempt complained about >> flock so I tried the posix-locks feature. >> >> ---jtg-server.conf-- >> volume jtg-ds >> type storage/posix >> option directory /opt/jtg/shared/jtg-ds >> end-volume >> >> volume jtg-ds-locks >> type features/posix-locks >> # option mandatory on >> subvolumes jtg-ds >> end-volume >> >> volume jtg-ns >> type storage/posix >> option directory /opt/jtg/shared/jtg-ns >> end-volume >> >> volume jtg-ns-locks >> type features/posix-locks >> subvolumes jtg-ns >> end-volume >> >> volume server >> type protocol/server >> option transport-type tcp/server >> option auth.ip.jtg-ds-locks.allow X.X.X.* >> option auth.ip.jtg-ns-locks.allow X.X.X.* >> subvolumes jtg-ds jtg-ds-locks jtg-ns jtg-ns-locks >> end-volume >> --jtg-server.vol >> >> [root@jtg111 conf]# more jtg-client.vol >> volume remote1 >> type protocol/client >> option transport-type tcp/client >> option remote-host jtg111.XXXX >> option remote-subvolume jtg-ds-locks >> end-volume >> >> volume remote2 >> type protocol/client >> option transport-type tcp/client >> option remote-host jtg113.XXX >> option remote-subvolume jtg-ds-locks >> end-volume >> >> volume mirror0 >> type cluster/afr >> subvolumes remote1 remote2 >> option self-heal on >> end-volume >> >> On both nodes: >> glusterfsd -f /opt/jtg/conf/jtg-server.vol >> glusterfs -L debug -f /opt/jtg/conf/jtg-client.vol /opt/jtg/mnt/shared/ >> >> At this point i have an two node system. Read/write from either side. >> All is well. Now my goal is to install a vserver onto the gluster >> volume. >> mkdir /opt/jtg/mnt/shared/etc/ >> mkdir /opt/jtg/mnt/shared/vservers/ >> >> ---attempt 1---- >> vserver glustertest build -m template --hostname >> glustertest.XXXXXX.com --interface eth0:10.0.0.2/24 --rootdir >> /opt/jtg/mnt/shared/vservers/ --confdir >> /opt/jtg/mnt/shared/etc/glustertest -- -d centos5 -t >> /vservers/.templates/test01.tar >> >> vcontext: open("/dev/null"): Permission denied >> >> An error occured while executing the vserver startup sequence; when >> there are no other messages, it is very likely that the init-script >> (bash -c >> exec > /dev/null >> exec 2> /dev/null >> trap 'kill -s 9 -- -1; exit 0' INT >> sleep 15 >> kill -s 15 -- -1 >> sleep 1 >> kill -s 9 -- -1) failed. >> >> Common causes are: >> * /etc/rc.d/rc on Fedora Core 1 and RH9 fails always; the 'apt-rpm' build >> method knows how to deal with this, but on existing installations, >> appending 'true' to this file will help. >> >> >> Failed to start vserver 'glustertest' >> 'vserver ... suexec' is supported for running vservers only; aborting... >> 'vserver ... suexec' is supported for running vservers only; aborting... >> 'vserver ... suexec' is supported for running vservers only; aborting... >> 'vserver ... suexec' is supported for running vservers only; aborting... >> 'vserver ... suexec' is supported for running vservers only; aborting... >> 'vserver ... suexec' is supported for running vservers only; aborting... >> 'vserver ... suexec' is supported for running vservers only; aborting... >> >> then i put a link in /etc/vservers >> ---end---- >> >> >> ---attempt 2--- >> vserver glustertest build -m template --hostname >> glustertest.XXXXX.com --interface eth0:10.0.0.2/24 --rootdir >> /opt/jtg/mnt/shared/vservers/ -- -d centos5 -t >> /vservers/.templates/test01.tar >> >> >> vshelper.init: can not determine xid of vserver 'glustertest'; >> returned value was '' >> >> This usually means that you're using an init-less init-style, but the >> guest isn't configured to start any service. Try enabling a service, >> changing the init-style, or making the contexts persistent. >> >> An error occured after executing the vserver startup sequence. This >> means that some processes may exist in the created context and the >> manual execution of >> >> /usr/sbin/vserver '/etc/vservers/glustertest' stop >> >> is recommended to fix this. >> ------------------------ >> So. The question. Should this work? Does this go back to the extended >> attributes we spoke of before. My goal is not to do any unification >> just an install. If anyone has ideas let me know! >> - -- Protect your environment - close windows and adopt a penguin! PGP-Key: 3B2C 8095 A7DF 8BB5 2CFF 8168 CAB7 B0DD 3985 1721 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIyTDcyrew3TmFFyERAtc0AJ429Uq/3thjvLe5hRnFx0CllOB3GQCdFGgQ KgbBI3642GfqoSJcRcXPYpg= =0k0t -----END PGP SIGNATURE-----