Jan 262012
 

In this “how to” we will go over installing Ubuntu 10.04 TLS on a Soekris Net4801 SBC (Single board computer).

To accomplish this, we will be network booting the Net4801 since it does not have any installation type storage (no cd-rom, and USB ports are not bootable), also since the Net4801 does not have a video card, or keyboard, we will be performing the installation over a serial console.

You can use this guide to perform the same function on other SBCs or other devices (even a standard server). The methods in the guide to both network boot, and provide a serial console are not mutual specific and can be done on their own (example, you don’t need to network boot to install using a serial console, or vice versa).

In this how to, we are using the Soekris Net4801 since it’s a small, interesting little computer which is designed as a somewhat open platform for router, wireless, and numerous other types of development and production type uses. The Net4801 specifications are available here: http://soekris.com/products/net4801.html.

The instructions I provide are using software and systems I have available to myself. Your environment may be different so remember that Google is your friend. The concepts will be the same.

Requirements

To get started, we need:

  • 1 X Soekris Net4801
  • 1 X Computer with a serial port
  • 1 X Serial DB-9 Null Modem Cable
  • 1 X Linux or Windows computer running TFTP server and web server

When network booting the Ubuntu installer, you can either install directly off the internet (which requires simply networking booting, and following the installation instructions) or you can provide the installer the installation files which may speed things up if you are on a slow connection. For the purpose of enlightening whoever is reading this, we are going to provide the installer the files.

How to Install Ubuntu over PXE netboot

1) Install a TFTP Server

The first thing we have to do is create the environment necessary to network boot the Net4801. In my case I have a CentOS 6 server. I installed the tftp server by issuing “yum install tftp-server*”. After this is complete we open up /etc/xinetd.d/tftp and change the disable value to no. Go ahead and restart xinetd by typing “/etc/init.d/xinetd restart”.

We now have a TFTP server providing everything inside of /var/lib/tftpboot.

2) Configure the DHCP Server to provide PXE boot info to PXE clients

In my case, I have a Windows Server 2008 R2 box providing DHCP to my network. I simply log on to the server, and open the DHCP Server GUI. I browse to my network scope, and right click on “Scope Options” and hit “Configure Options”. We need to specify two options: First is “066 Boot Server Host Name” which we set to the IP address of the TFTP Server, and second “067 Bootfile Name” which we set to “pxelinux.0”. That’s it! When the PXE client boots it will receive this information.

3) Configure netboot files

If you have the Ubuntu alternative CD, you can copy over everything inside of the install/netboot directory to /var/lib/tftpboot.

If you don’t have the Ubuntu CD, change your working directory to /var/lib/tftpboot, and type “wget ftp://ubuntu.arcticnetwork.ca/pub/ubuntu/ubuntu/dists/lucid/main/installer-i386/current/images/netboot/netboot.tar.gz”. After this, type “tar zxvf netboot.tar.gz”. This will extract the netboot components to the directory.

When this is completed, this will provide the network installer for Ubuntu. Since we are using a serial terminal to install Ubuntu on the Net4801, we now have to configure the bootloader, and installer to use the serial console.

Inside of the /var/lib/tftpboot directory, open pxelinux.cfg/default using your favorite editor. Add these two lines to the top of the file:

console 0
serial 0 19200 0

Save and close. We have just instructed the bootloader to use the first serial port on the system to provide console. Next we need to configure the kernel to input/output to the serial port aswell.

Open ubuntu-installer/i386/boot-screens/text.cfg in your editor. We are going to remove and add a few things to the “append” line under the “install label”. We are going to remove the word quiet, and replace it with console=ttyS0,19200. After completed it should look like this:

default install
label install
menu label ^Install
menu default
kernel ubuntu-installer/i386/linux
append vga=normal initrd=ubuntu-installer/i386/initrd.gz -- console=ttyS0,19200 earlyprint=serial,ttyS0,19200

Now that this is complete, the kernel will now input/output to the serial console.

4) Install sources (you can skip this, but please read)

If you are installing from the internet, you can simply skip this step. If you have the Ubuntu alternative CD, or CD Image, and want to install from those sources, this is what we need to do. In my case, I had the ubuntu .iso file.

On my CentOS server, I have Apache httpd installed. I have the .iso file in /root/. I type “mkdir /var/www/html/mount” to create a directory called mount inside of the web root. I then make sure I’m in /root/ when typing “mount cd-image-name.iso /var/www/html/mount -o loop” which mounts the CD Image to the mount directory.

We have now successfully mounted the CD image to the web server.

5) Network Boot the Soekris Net4801 via PXE

We now have the environment configured, it’s finally time to network boot the Net4801. Keep in mind, with a serial connection, one of the only problems you’ll run into is a) configuring software (ie. Linux, Grub, Bootloders) to use it, and b) speed settings. We’ve addressed the first issue already with configuration files, however we need to setup speed values on both the BIOS for the Net4801, and speed value for the client (in my case PuTTY). While you can use a whole range, I like to use 19200. It’s friendly, and I never have any issues 🙂

Hook up the Net4801 to your computer’s serial port using your Null Modem Serial cable. Open up PuTTY, and instead of using SSH, use Serial, and set the speed to 19200. I beleive this is the default for a fresh out of the box Net4801, and start the connection. Power on the Net4801 and you should see the startup text.

So right now, the connection is working, but I thought I’d go over a few things. Hit Ctrl + P while the BIOS is posting, and type show.

These are variables you can configure on the Net4801. A few to remember are: a) ConSpeed – Serial port speed, has to match on both sides b) Flash – Either Primary or Secondary, this specifies whether it is Master or Slave on the IDE Channel, just like traditional older IDE based computers. c) BootDrive – This is standard booting order, 80 = IDE Master, 81 = IDE Slave, F0 = PXE Network boot.

Anyways, that’s that. So now we want to network boot. While in the ComBios command console, type in “Boot F0”. This will initiate the network boot. Things might look a bit weird at first, however eventually it will prompt you for something, simply type “install” and hit enter. After the kernel boots, the Ubuntu text installation should start. Now it’s easy and normal.

FINAL NOTES

The Soekris Net4801’s processor is a i586 class processor. Ubuntu dropped support for i586 as of 10.10 and later. This is why I chose 10.04 LTS.

There are issues with the installer on the Net4801. Issue being, after specifying network configuration, once it starts to download the initial installer components, the installer will freeze. Usually the screen goes blank for up to 45 minutes when it is working, however when this issue occurs, it will permanently freeze. 10 Months ago I narrowed down what was causing this, however have forgotten. I think it has something to do with just having a IDE drive connected to the Soekris, I think it started working when switching to Compact Flash for internal storage.

POST-INSTALL CONFIGURATION

After installing, I noticed a bunch of weird things like ureadahead and plymouth crashing on startup (due to lack of resources). Also, some things were not showing up on the serial console which I wanted to (this is because everyone want’s GUI’s these days).

Few recommendations on cleaning up your install:

1) Disable AppArmor – It’s a waste of resources

Type “sudo update-rc.d -f apparmor remove”

2) Configure GRUB – Get the proper stuff going to the console

Open /etc/default/grub in your favorite editor, remove everything and paste this:

GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=3
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="noplymouth text nosplash"
GRUB_CMDLINE_LINUX="console=ttyS0,19200 earlyprint=serial,ttyS0,19200"
GRUB_TERMINAL=console
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=19200 --stop=1"

That disables plymouth, configures a text console, and shows GRUB on boot. After this is done, run “update-grub” and “update-initramfs -u”.

3) Set time – I don’t know why but my system lost it’s time, maybe my battery is going, for some reason this wasn’t automatic.

Type “ntpdate time.windows.com”

4) Compiler and Building stuff – If you’re going to compile anything, run this, or skip this

Type “apt-get install fakeroot build-essential libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev”

Have any questions, feel free to leave a comment.

Oct 142011
 

In this tutorial, I will be showing you how to get Lio-Target (an iSCSI target that is compatible with persistent reservations required by both VMware and MS Clustering) running on CentOS 6.

While this tutorial is targetted for CentOS 6 users, I see no reason why this should work on any other newer distributions.

Please note that while Lio-Target 4.x (and required tcm_loop and iSCSI) is available on newer/non-stable development kernels, Lio 3.X is stable, and currently builds nicely on CentOS 6. I will be doing up a tutorial for Lio 4.X once I myself start using it.

One more note, In the past I have thrown up a few tutorials on how to get Lio-Target running on various Linux distributions. These tutorials have worked for some, and not for others. I myself have had a few difficulties replicating the success I did originally. I myself am a technical guy, I do not understand some developer terms, and am not an expert in understanding some development cycles. This is one of the reasons why I had so many difficulties earlier. Since the earlier tutorials, I have caught up to speed and am familiar with what is required to get Lio-Target running.

Now on to the tutorial:

It is a good idea to start with a fresh install of CentOS 6. Make sure you do not have any of the iSCSI target packages installed that ship with CentOS. In my case I had to remove a package called something like “iSCSI-Target-utils” (This shipped with the CentOS 6 install).

1. Let’s download the software. We need to download both the 3.5 version of Lio-Target, along with Lio-utils which was built for 3.x of Lio-Target. (I chose the RisingTide Systems GIT repo since lio related projects have been missing from kernel.org’s GIT repo due to the issues kernel.org has been having recently).

Issue the following commands:

git clone git://risingtidesystems.com/lio-core-backports.git lio-core-backports.git

git clone git://risingtidesystems.com/lio-utils.git lio-utils.git

cd lio-utils.git/

git checkout --track -b lio-3.5 origin/lio-3.5

cd ..

(You have now downloaded both Lio-Target 3.5 backport, and lio-utils for lio-target 3.x)

2. Build kernel modules for your existing running CentOS kernel. Make sure you change in to the lio-core-backports directory first.

Change in to the lio-core-backports directory then issue the following commands:

make

make install

(You have now built, and installed the kernel modules for Lio-Target)

3. Build lio-utils and install. This is one of the tasks I had difficulties with, for some reason the install scripts were calling out to the incorrect python directory, I found a fix to this myself.

Apply the fix first:

Go into the tcm-py and lio-py directories inside of the lio-utils directory. Open the install.sh in both the tcm-py and lio-py directories and change the “SITE_PACKAGES” string to reflect the following:

SITE_PACKAGES=/usr/lib/python2.6/site-packages

Remember to do this in both the install.sh files for lio-py and tcm-py. Now on to building and installing lio-utils.

Issue the following commands from the lio-utils directory:

make

make install

And you are now done!

Lio-Target and Lio-Utils have no succesfully been installed. As you can see, this was way easier than my previous tutorials, and doesn’t include and rebuilding of kernels, etc… One of the plus’s is that you actually build the kernel modules for the existing CentOS kernel.

One last thing. Start lio-target by issuing the command:

/etc/init.d/target start

And do a ‘dmesg’ to confirm that it started ok!

As always, feel free to post any comments or questions. I’ll do my best to help!

Jan 102011
 

I notice quite a bit of traffic coming in (alot of it is the same people coming back) searching for information on VMware vSphere using iSCSI, specifically Lio-Target (because of the compatibility with SCSI persistant reservations).

In the past I’ve been jumping all over the place testing Lio-Target on different distrobutions, test scenarios, etc… I’ve officially implemented it into my production network and just wanted to report it’s been running solid for a few months now.

Current Working Configuration (Stable):

Currently at the moment I have numerous HP Servers (ML350s, DL360s) running ESXi off an internal usb-key. These ESXi hosts are accesing numerous iSCSI targets over gigabit hosted on a DL360 G6 with 2 X MSA20 storage units. The server hosting the storage is running Ubuntu 10.10 and has been rock solid with absolutely no issues. I’m fully utilizing VMotion amongst all the hosts, and all hosts have concurrent access to the iSCSI targets. This is running in full production and I’m fully confident in the configuration and setup.

Future Plans:

Lio-Target is going upstream in to the Linux kernel on the next release (2.6.38). With the testing I did (and blogged about) in the past months, I have not been able to get the newer versions of Lio-target running stable on CentOS. Once a new version of CentOS is released, or there is a kernel upgrade available to bring CentOS to 2.6.38, I will be installing CentOS on to the storage server and adding more disk space. Once this change is complete, that will conclude any future changes for a while (excluding adding more ESXi hosts).

If anyone has any questions on my setup, or configuration with something similar, or have any issues with Lio-Target, please feel free to leave a comment and I’ll see if I can help!

Oct 202010
 

Well, my Ubuntu Server box running Lio-Target is still running great, and performing perfectly under the continous stress testing I’ve done.

While I’m waiting for a few more days of the stress test to finish, I’m setting up my old Soekris Net4801.

I’m install Ubuntu Server 10.04 TLS on to the Soekris Net4801 via remote PXE netboot. Afterwards I’m going to compile Lio-target 3.4 (kernel 2.6.34) on the device and test out performance of the iSCSI target. It won’t be anything special since the net4801 is so slow, but it’ll be interesting to see for sure.

I’ll also be sticking in a PCI – USB2.0 card inside of the net4801 to get USB2.0 speeds on a removal drive.

After this little experiment I might rip out the cross compiler and build Lio-Target on a Linksys WRT610N if I can to check out the performance on that. I know these little devices have quite a bit of power, gigabit networking, and a single USB 2.0 port built right in!

As promised I’ll be posting more detailed posts in the future once all the fun is done!

Oct 052010
 

PLEASE VISIT http://www.stephenwagner.com/?p=300 FOR AN UPDATED TUTORIAL.

 

Disclaimer: Please note that whenever doing any of the steps mentioned in this post, if you do not know what you are doing, you can render your linux install usless. Please do NOT use this in a production environment, and only use for testing. I’m not liable if you toast your linux install.

First and foremost, you need to know that from what I understand, the typical kernels that ship with CentOS have numerous patches, modifications, and updates from the typical Linux kernel releases. This is one reason why the releases of the kernels are always behind compared to ones that are actively being developed. I could be wrong, but I think the Redhat kernel patches are applied to CentOS kernels.

This tutorial will walk you through on how to get you up and running in the dirtiest and quickest method. You may and probably will have to modify things, re-compile, etc… to resolve any issues you may run in to.

I wrote this article only to help more people get up and running with one of the only open-source Linux iSCSI targets that has been certified by VMware (Certified when running on certain appliances) to be used in a vSphere environment (supports Vmotion, SCSI reservations, etc…)

I’m going to assume you have CentOS 5.5 installed and is fully up to date.

Anyways, here’s a breakdown of what we will be doing to get Lio-Target to run on CentOS:

  • Download lio-target modified kernel
  • Download lio-utils
  • Compile a modified kernel using existing kernel .config file
  • Compile and install lio-utils
  • Sample commands to setup a dedicated drive on your system as a iSCSI target (to test)
  • Compose two quick and dirty config files so lio-target can run

Download the lio-target modified kernel

For this step, you will need to have git installed. From what I understand git is not an option during the CentOS install, and cannot be installed using the default typical yum repos. To get git installed, we will first add the “RPMFusion” and “EPEL” yum repos.

(Info on how to install these can be found at http://rpmfusion.org/ and https://fedoraproject.org/wiki/EPEL)

Once you have installed these, it’s time to install git. This can be easily done by typing:

yum install git

After git is installed, it’s now time to download the lio-target kernel.

Type in:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/nab/lio-core-2.6.git lio-core-2.6.git

It is now downloading the source. After this is complete, change directory to the new directory that was created, and type in:

git checkout origin/lio-3.4

You now have the source.

Download lio-utils

This is a pretty simple step. If you just completed the step above, make sure you cd back in to a directory that you can use as a workspace, and make sure you do NOT do this, inside of the directory that was downloaded above. Type in:

git clone git://git.kernel.org/pub/scm/linux/storage/lio/lio-utils.git lio-utils.git

This should create a directory called “lio-utils.git”. This step is done.

At this point

You should have two directories inside of your workspace:

lio-core-2.6.git

lio-utils.git

Compile a modified kernel using existing kernel .config file

It is now time to build a kernel. First of all, issue a uname –a and note the kernel you are running. In my case it’s “2.6.18-194.17.1.el5”. I’m going to take a config file from the boot directory that matches the number, and copy it to the lio-core-2.6.git directory, only change the name to “.config”. This is what I would type in on my end:

cp /boot/config-2.6.18-194.17.1.el5 /root/lio-core-2.6.git/.config

This command above would copy the config for the CentOS kernel, and move it to the lio-core-2.6.git directory with the new name of “.config”. Now change into the lio-core-2.6.git directory:

cd lio-core-2.6.git

At this point we are going to run a command to help the config adjust to the newer kernel version. Type in:

make oldconfig

This will spawn numerous messages. You can just go ahead and keep hitting enter. It will seem like an endless loop, however eventually it will complete. Now the next step, this is important. By default, new kernels use a different sysfs structure. We need to turn on the depreciated item that CentOS uses. Type in:

make menuconfig

Navigate to:

“General Setup”, then check (put a star in the box) for “enable deprecated sysfs features to support old userspace tools”.

Now hit tab to select exit. Once again tab to exit at the main menu. We are now ready to compile! To do a quick compile, type in:

make && make modules && make modules_install && make install

Feel free to disapeer for a while. This will take some time depending on the performance of your system. Once done, you now have a kernel with lio-target built in that is compiled and installed.

Keep in mind that this is NOT the default kernel, and you will have to select this to boot when starting your system. To change this, modify /etc/grub.conf and change the default value to whichever item it is (remember that the first item is 0, and not 1).

Let’s boot the new kernel. Safely shut down your linux box, reboot, and when grub shows up, boot using the new kernel you compiled.

Please note: This guide is a quick and dirty way to get this up and running. Since we skipped customizing the kernel, the kernel you compiled will no doubt come up with errors on boot. I simply ignore this. You can too, just to get this up and running. You can come back at a later time and refine your custom kernel to be used.

Compile and install lio-utils

This is simple. Change directory to the directory where you downloaded lio-utils.

cd /root/lio-utils.git

And to compile and install:

make

make install

And BAM, you’re done, that was easy!

Sample commands to setup a dedicated drive on your system as a iSCSI target (to test)

It’s show time. I’m still fairly new to the configuration and usage of lio-target, so I’m just posting the commands to get it working on a dedicated disk. There are no manual (man page) entries installed by lio-utils, so you will need to use lio_node –help and tcm_node –help for more information on proper usage.

Keep in mind, DO NOT use your linux disk as a iSCSI target! Let’s pertend we have a second disk in the system /dev/sdb that we want to turn in to a target. This is what we would type:

tcm_node –block iblock_0/array /dev/sdb

/etc/init.d/target start

lio_node –addlun iqn.2010.com.stephenwagner.iscsi:array 1 0 iscsi00 iblock_0/array

lio_node –listendpoints

lio_node –addnp iqn.2010.com.stephenwagner.iscsi:array 1 192.168.0.10:3260

lio_node –listendpoints

lio_node –disableauth iqn.2010.com.stephenwagner.iscsi:array 1

lio_node –addlunacl iqn.2010.com.stephenwagner.iscsi:array 1 iqn.CLIENTINITIATORHERE.com 0 0

lio_node –enabletpg iqn.2010.com.stephenwagner.iscsi:array 1

The above will create a target, and discovery portal for /dev/sdb on 192.168.0.10. This will also disable CHAP authentication, and will allow the initator I specified above to connect.

Please change /dev/sdb to the drive you want to use

Please change iqn.2010.com.stephenwagner.iscsi:array to the iSCSI target you want to call yours.

Please change 192.168.0.10 to the IP of the iSCSI target your configuring. Leave port as 3260.

Please change iqn.CLIENTINITIATORHERE.com to the iqn for your initiator (client). This will be set on the client you are using to connect to the iSCSI target.

BAM, your target is up and running! Keep in mind, this configuration is lost upon reboot.

Compose two quick and dirty config files so lio-target can run

Here’s what you need to put in the config files to make the above config work on boot:

/etc/target/tcm_start.sh should contain:

tcm_node –block iblock_0/array /dev/sdb

/etc/target/lio_start.sh should contain:

lio_node –addlun iqn.2010.com.stephenwagner.iscsi:array 1 0 iscsi00 iblock_0/array

lio_node –addnp iqn.2010.com.stephenwagner.iscsi:array 1 192.168.0.10:3260

lio_node –disableauth iqn.2010.com.stephenwagner.iscsi:array 1

lio_node –addlunacl iqn.2010.com.stephenwagner.iscsi:array 1 iqn.1991-05.com.CLIENTINITATOR.com 0 0

lio_node –enabletpg iqn.2010.com.stephenwagner.iscsi:array 1

After you make these config files. You should be able to start lio-target in a running state, by issuing:

/etc/init.d/target start

And remember, that you can always view a live feed of what’s going on by issuing:

tail –f /var/log/messages

Hope this helps!