Hi all,
since few days I'm trying hekafs and i really like it. Neverthless, I found some issues wich I described below.
The configuration I used is the following :
- 4 (virtualized ) servers named glusterfs1,2,3,4 all running centos (I use the epel repo for the rpms) or all running fedora 16
- 1 (virtualized) client named glusterfs5 running centos or fedora
- the host runs fedora 17 and the network was bridged for all the virtual machines.
My goal was to achieve a "replicated" (-r 2) hekafs with ssl. Each server having its own key and public certificate.
The first problem I had was with documentation :
- man 8 hekafs :
for creating keys, it is written "openssl genrsa 1024 -out server.key". In fact the number of bits must be at the end of the command or the output file will not be created so the line has to be "openssl genrsa -out server.key 1024".
- following the link given in fedora wiki (https://fedoraproject.org/wiki/Features/CloudFS) I have access to the file named README.ssl.
But when I cloned the git repository, this file does not exist. I tried to search in the git historybut I can not find it (but my knowledge of git is (very) low so I could have missed something).
For my configuration, I got the information that the different files server*.pem must be concatened in a file named "root.pem". But, this information is not in the man of hekafs. So does the file containing the different (server) certificate must be named "root.pem" or can we set another name for it ?
- from the hfs_mount manpage, I was not able to use the data key parameter. Does this parameter work only with branch aes from the git repository ?
During my testing, maybe the most annoying thing was that if something went wrong during a hfs_mount command, no information that a problem occured is given.
For example, if you launch a hfs_mount command with a wrong password or bad client's certificate, you always get 0 with echo $? . The log file contains something like this :
"
[2012-06-01 13:37:39.634914] E [graph.c:526:glusterfs_graph_activate] 0-graph: init failed
[2012-06-01 13:37:39.635219] W [glusterfsd.c:727:cleanup_and_exit] (-->/usr/sbin/glusterfs(main+0x295) [0x405e85] (-->/usr/sbin/glusterfs(glusterfs_volumes_init+0x145) [0x404d45] (-->/usr/sbin/glusterfs(glusterfs_process_volfp+0x198) [0x404bf8]))) 0-: received signum (0), shutting down
[2012-06-01 13:37:39.635290] I [fuse-bridge.c:3727:fini] 0-fuse: Unmounting '/gluster/'.
"
So fuse silently unmounts "/gluster" (which succeeds), and $? contains then the 0 value instead of a code error. Maybe, in this case the function which unmounts the folder should "remember" that something went wrong ?
I also had other issues but I suspect that it is rather due to my ignorance of hekafs :) So I prefer to ask if I observed "normal" behaviours :) :
- when I want to do a hfs_add_node (cli), I get a SSH error "The authenticity of host '192.168.1.199 (192.168.1.199)' can't be established". So I create a ssh key and copy it to the host by a ssh-copy-id command and after that the hfs_add_node works without any problem.
-in order to test the self-healing facility, I shutted down a server and copy some file from the client. When i turn back on the server, the "missing" files do not appear (I waited 10 minutes since I read that glusterfs runs a self-healing every 600s). On the client, I need to unmount and mount the folder (then files appears on the server but have 0 size) and then launch a "find /gluster/ -noleaf -print0 | xargs --null stat >/dev/null" for "really" having the files on the server.
- I try to compile hekafs from source via the fedora-ize script but it fails. It seems that some folders have beem renamed (pkg->packaging ?). A make fedora in the packaging folder does the trick.
- each server is running django at the 8080 port. But I could not find a way to turn it off (except by a iptable rule). I think it could be a security problem because a local attaker can run an nmap scan on then local network and then can have access to the configuration of the servers.
- In glusterfs, you could share files via acls to "few people" . Is it possible to do such kinds of thing with different tenants ? ( I suspect the answer to be no since on the server side each tenant has its own folder).
I am writing some documentation of my setup. Do you think it could be interesting to other users?
Sorry for this quite long mail and many thanks to all for this great peace of software .
Best regards,
Yves
--
The code also assumes that it's difficult to misspell "a" or "b". :-)
-- Larry Wall in <199710221731.KAA24396(a)wall.org>
