Myself

By , May 31, 2015 3:23 pm

Myself

 

Myself
蒋云舒,4 岁作

涂鸦

By , May 19, 2015 3:51 pm

cat

 

小猫
蒋云舒,4 岁做

 

sheep

 

小羊
蒋云舒,4 岁做

Distributed File System on Amazon Linux — GlusterFS

By , May 13, 2015 9:35 am

[Introduction]

This article provides a quick started guide on how to set up and configure GlusterFS on Amazon Linux. Two EC2 instance are being launched to accomplish this goal. On both EC2 instances, there is an instance-store volume serving as the shared storage.

Edit /etc/hosts on both EC2 instances with the following entries (assuming that the private IP addresses are 172.31.0.11 and 172.31.0.12).

172.31.4.11	gluster01
172.31.4.12	gluster02

Create the a repository definition /etc/yum.repos.d/gluster-epel.repo with the following content:

# Place this file in your /etc/yum.repos.d/ directory

[glusterfs-epel]
name=GlusterFS is a clustered file-system capable of scaling to several petabytes.
baseurl=http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/epel-6/$basearch/
enabled=1
skip_if_unavailable=1
gpgcheck=0

[glusterfs-noarch-epel]
name=GlusterFS is a clustered file-system capable of scaling to several petabytes.
baseurl=http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/epel-6/noarch
enabled=1
skip_if_unavailable=1
gpgcheck=0

[glusterfs-source-epel]
name=GlusterFS is a clustered file-system capable of scaling to several petabytes. - Source
baseurl=http://download.gluster.org/pub/gluster/glusterfs/LATEST/EPEL.repo/epel-6/SRPMS
enabled=0
skip_if_unavailable=1
gpgcheck=0

Use the following commands to install the necessary packages and start services.

sudo yum update
sudo yum install fuse fuse-libs nfs-utils
sudo yum install glusterfs glusterfs-fuse glusterfs-geo-replication glusterfs-server

sudo chkconfig glusterd on
sudo chkconfig glusterfsd on
sudo chkconfig rpcbind on
sudo service glusterd start
sudo service rpcbind start

[Configurations]

On both EC2 instances, create file system on the instance-store volume and mount it to /glusterfs.

sudo mkdir /glusterfs
sudo mkfs.ext4 /dev/xvdb
sudo mount /dev/xvdb /glusterfs
sudo mkdir -p /glusterfs/brick

On gluster01, probe the other EC2 instance and create the GlusterFS volume. The name of the volume is “test-volume”.

sudo gluster peer probe gluster02
sudo gluster volume create test-volume replica 2 transport tcp gluster01:/glusterfs/brick gluster02:/glusterfs/brick
sudo gluster volume start test-volume

On both EC2 instance, mount the GlusterFS volume. The name of the volume is “test-volume”. Then we change the ownership of the /mnt/glusterfs folder to “ec2-user:ec2-user” so that the default “ec2-user” can use the shared folder directly.

sudo mkdir -p /mnt/glusterfs
sudo mount -t glusterfs gluster01:test-volume /mnt/glusterfs
cd /mnt
sudo chown -R ec2-user:ec2-user glustefs
mount
df -h

Now the shared file system has been set up, and you can create a text file under /mnt/glusterfs and observe that the file created appears on both EC2 instances. Please bear in mind that this is only a quick start guide, and you should not use this configuration directly in a production system without further tunings.

Distributed File System on Amazon Linux — MooseFS

By , May 13, 2015 6:40 am

[Introduction]

This article provides a quick started guide on how to set up and configure MooseFS on Amazon Linux. Two EC2 instance are being launched to accomplish this goal. On both EC2 instances, there is an instance-store volume serving as the shared storage. One of the EC2 instance is being used as the master server, while both EC2 instances are chunk servers (storage servers).

[Master Server Installation]

Edit /etc/hosts, add the following record (assuming that the private IP of the master node is 172.31.0.10):

172.31.0.10    mfsmaster

Then run the following commands to add the MooseFS repository:

wget http://ppa.moosefs.com/stable/yum/RPM-GPG-KEY-MooseFS
sudo cp RPM-GPG-KEY-MooseFS /etc/pki/rpm-gpg/
wget http://ppa.moosefs.com/stable/yum/MooseFS.repo
sudo cp MooseFS.repo /etc/yum.repos.d/

Install MooseFS master and CLI:

sudo yum update
sudo yum install moosefs-ce-master
sudo yum install moosefs-ce-cli
cd /etc/mfs

In the /etc/mfs folder, you should see mfsmaster.cfg and mfsexports.cfg. If they don’t exist, copy mfsmaster.cfg.dist to mfsmaster.cfg and copy mfsexports.cfg.dist to mfsexports.cfg.

Modify mfsexports.cfg to add permission for the 172.31.0.0/16 subnet. Add this line to the end of the file:

172.31.0.0/16        /    rw.alldirs,maproot=0

One more CRITICAL modification:

cd /var/lib/mfs
sudo cp metadata.mfs.empty metadata.mfs

Start MFS master

sudo service mfsmaster start

At this point, the MooseFS master node is running successfully. You can use the following MFS CLI to do a quick check:

mfscli -SIN

[Chunk Server Installation]

Edit /etc/hosts, add the following record (assuming that the private IP of the master node is 172.31.0.10):

172.31.0.10    mfsmaster

Then run the following commands to add the MooseFS repository:

wget http://ppa.moosefs.com/stable/yum/RPM-GPG-KEY-MooseFS
sudo cp RPM-GPG-KEY-MooseFS /etc/pki/rpm-gpg/
wget http://ppa.moosefs.com/stable/yum/MooseFS.repo
sudo cp MooseFS.repo /etc/yum.repos.d/

Install MooseFS chunk server and CLI:

sudo yum update
sudo yum install moosefs-ce-chunkserver
sudo yum install moosefs-ce-cli
cd /etc/mfs

In the /etc/mfs folder, you should see mfschunkserver.cfg and mfshdd.cfg. If they don’t exist, copy mfschunkserver.cfg.dist to mfschunkserver.cfg and copy mfshdd.cfg.dist to mfshdd.cfg.

Assuming that you have a second EBS volume (or instance-store volume) on your EC2 instance, which is /dev/xvdb, that you would like to use as the storage for MooseFS.

sudo mkfs.ext4 /dev/xvdb
sudo mkdir /mfs
sudo mount /dev/xvdb /mfs
sudo chown -R mfs:mfs /mfs

Then edit /etc/mfs/mfshdd.cfg, add one line to the end of the file:

/mfs

Create a file /etc/default/moosefs-ce-chunkserver, with the following content:

MFSCHUNKSERVER_ENABLE=true

Start MooseFS chunk server with the following command:

sudo service mfschunkserver start

Now you can check the status of the whole storage system using MooseFS CLI:

mfscli -h
mfscli -SCS

[Client Installation]

Edit /etc/hosts, add the following record (assuming that the private IP of the master node is 172.31.0.10):

172.31.0.10    mfsmaster

Then run the following commands to add the MooseFS repository:

wget http://ppa.moosefs.com/stable/yum/RPM-GPG-KEY-MooseFS
sudo cp RPM-GPG-KEY-MooseFS /etc/pki/rpm-gpg/
wget http://ppa.moosefs.com/stable/yum/MooseFS.repo
sudo cp MooseFS.repo /etc/yum.repos.d/

Install MooseFS client and CLI:

sudo yum update
sudo yum install fuse
sudo yum install moosefs-ce-client
sudo yum install moosefs-ce-cli

Actually mount the MooseFS shared storage to local computer:

sudo mkdir -p /mnt/mfs
sudo mfsmount /mnt/mfs -H mfsmaster
df -h

Now the shared file system has been set up, and you can create a text file under /mnt/glusterfs and observe that the file created appears on both EC2 instances. Please bear in mind that this is only a quick start guide, and you should not use this configuration directly in a production system without further tunings.

HP Public Cloud — Internal Server Error

By , May 5, 2015 6:35 pm

屏幕快照 2014-05-06 上午9.46.21

This is a screen capture taken off my computer on May 06, 2014. The second day this screen shot — along with a detailed explanation — was sent to a couple of folks working at HP Cloud. Today is the first one-year anniversary of this screen capture, and I think it is a good time to tell the story behind it.

In February 2014, my family relocated from China to Australia. As you can imagine, I have a new local bank account with a new bank card, and my old bank card in China gradually expired. I managed to update my payment information with most of the online services that I use. However, with HP Cloud, each and every time I got this unfriendly message when submitting my payment information through the Horizon console. On May 06 2014 I received an email titled “Action Required – Payment Declined” from HP Cloud, which was very frustrating for a customer with the good intention to pay the bill and had tried his best to give the vendor the required payment information. I ended up initiating an online chat session with the HP Cloud support team, and the agent happily updated my payment information and resolved the issue. I understand that there is usually a gap between support and product team, so I took the extra effort to send the screen capture to a couple of  folks with HP Cloud email addresses out of my address book. A director level manager replied to say thank you, and pointed out that the issue had been escalated to public cloud billing.

Since then, I got this “Action Required – Payment Declined” email notification almost every month (but not every month). Each time I got this message, I attempted to update my payment information again, but was greeted with the same “Internal Server Error”. Then I ended up initiating another chat session with the HP Cloud support team, who happily resolved the issue for me.

On April 08 2015, I was in another chat with the HP Cloud support team. I told the agent that I was upset by the situation, and the agent told me that “I’m sorry for the incontinence. Some cards you used are declined by your bank and you get an automatic notice from our system. This is a bank issues.” I was very unhappy about this accusation – I had tried my best to give HP Cloud my payment information, every month I reminded HP Cloud of the correct payment method, but HP Cloud insisted on using the wrong payment information. So I asked the agent for a transcript of this particular chat. I also told the agent that should this happen again in May, I was going to write a story and publish it to the Internet. The second day, after obtaining approval from the management, the agent sent me the transcript of the chat.

Today, I have a new “Action Required – Payment Declined” message waiting in my inbox. As promised last month, I write down this story, and I will present the story to the next HP Cloud support agent. I hope that he/she will enjoy reading it, and help me resolve the issue I have.

Panorama Theme by Themocracy