Friday, January 2, 2015

Unionfs an alternative to AUFS

Overview:



This is in continuation to my article on 'Read only root filesystem using AUFS'. Unionfs is a Unification File System in Linux can be seen as alternative to AUFS, philosophically trying to solve similar kind of problems.

The changes were easy to incorporate into my existing build set up compared to AUFS. But the forum seems to be not as active .I never got any response to my post.

Contents of init file .

#init file    
#!/bin/busybox sh

# Mount the /proc and /sys filesystems.
mount -t proc none /proc
mount -t sysfs none /sys

busybox --install -s

mknod /dev/null c 1 3
mknod /dev/tty c 5 0

# Select partitions here. This hardcodes the partitions inside the initramfs. It is also possible to parse
# kernel's command line .
echo "This script mounts rootfs RO with an unionfs RW layer."
mdev -s

#create mount folders in initramfs
mkdir -p /mnt/rw /mnt/ro
# Mount the partitions
# 1) mount the ro partition
ubiattach /dev/ubi_ctrl  -m 4 -O 2048
mount -t ubifs -o ro,noatime  ubi0:rootfs  /mnt/ro
# 2) mount the rw partition, it could be any writable filesystem in the current case it is tmpfs
mount -t tmpfs -o size=30M        tmpfs  /mnt/rw
   
   

# 3) mount the writable overlay to the static image
mount -t unionfs -o dirs=/mnt/rw=rw:/mnt/ro=ro none /mnt/union

# Clean up.
mkdir -p /mnt/union/rw /mnt/union/ro
mount --move /mnt/rw /mnt/union/rw
if [ $? -ne 0 ]; then
echo    root-unionfs error:      Failed to move /mnt/rw /mnt/union/rw
exit 0
fi

mount --move /mnt/ro /mnt/union/ro
if [ $? -ne 0 ]; then
echo    root-unionfs error:      Failed to move /mnt/ro /mnt/union/ro
exit 0
fi

umount /proc
umount /sys
mount -t devtmpfs none   /mnt/union/dev
if [ $? -ne 0 ]; then
echo    root-unionfs error:      Failed to mount  devtmpfs   /mnt/union/dev
exit 0
fi
# Boot the real thing
exec switch_root /mnt/union /sbin/init

echo "Failed to switch_root, dropping to a shell"
exec /bin/sh


patch kernel 2.6.37 with unionfs patch


Get and patch from the below link.
http://download.filesystems.org/unionfs/unionfs-2.x-latest/unionfs-2.6_for_2.6.37.6.diff.gz



#Extract the patch and navigate to the kernl directory.
Gunzip unionfs-2.6_for_2.6.37.6.diff.gz
cd /'PATH of TI linux'/kernel  ;

#Now Patch the TI kernel
patch -p1 /'PATH of unionfs patch file'/unionfs-2.6_for_2.6.37.6.diff
    

Kernel menuconfig captured in .config with respect to unionfs


CONFIG_INITRAMFS_SOURCE="initramfs.cpio"
CONFIG_INITRAMFS_ROOT_UID=0
CONFIG_INITRAMFS_ROOT_GID=0
CONFIG_INITRAMFS_COMPRESSION_NONE=y
CONFIG_UNION_FS=y
CONFIG_UNION_FS_XATTR=y
CONFIG_UNION_FS_DEBUG=y  




Create intramfs.cpio archive as mentioned in  folder and create an archive file and copy into the kernel source.

Recompile the kernel with initramfs and flash it on the kernel partition of the board, now reboot the  system with unionfs file system.

To make changes in read-only directory


mount -o remount,rw /ro
mount -t unionfs -o remount,rw unionfs /

Read only root filesystem using AUFS

Overview:


My prototype based on ti8168 evaluation board encounters frequent filesystem corruption due power outrage.  After looking on resources available on the web it was better if we have a read only root file system.articles on raspberry pi sited gave me a useful pointers to start my work on AUFS.my experience with AUFS mailing list has been overwhelming,would appreciate the maintainers for his prompt reply to all my query.

The idea is to implement a Read only root file-system using a AUFS support in kernel, AUFS is a filesystem  hosts read-only and read-write directories as AUFS file-system branches. currently the system incorporates two layers,the read-write layer will happen on tmpfs and read-only layer will host the ubifs root file system.My board support Linux kernel 2.6.37,so the discussion is with respect to this source.
 

Steps involved.


1. create a initramfs.  
2. patch the kernel with AUFS support

About initramfs

This is basic minimal root file system embedded in kernel, the kernel checks for the presence of the initramfs, if found, mounts it as / and runs /init. The init program is a shell script, in our case do the preparation to mount the read-write and read-only branches on Unification File System and finally give the complete control to real root file system.

The initramfs has a complete set of directories similar to a normal root file system. The busybox binary is statically build and copied to  /bin directory of initramfs. This is eventually available as cpio archive placed in the kernel source folder, this initial file system gets loaded into memory during the Linux boot process, the path of the archive is notified in the .config file of the kernel.

Initramfs creation.



#create initramfs folder to start the activity
mkdir initramfs
cd initramfs

# create required directories
mkdir -p aufs bin dev etc lib proc rootfs rw sbin sys usr/{bin,sbin}

# populate dev
mknod -m 622 dev/console c 5 1
mknod -m 622 dev/tty0 c 4 0

# Copy statically build busybox to initramfs
cp /bin/busybox bin/
ln -s busybox bin/sh

# create init file
cat > init <#!/bin/sh

echo "Waiting"
sleep 1

echo "Mounting dev proc sys"

mount -t devtmpfs none /dev
mount -t proc proc /proc
mount -t sysfs sysfs /sys

echo "Mount rootfs om mtd4  of raw NAND on /rootfs"
ubiattach /dev/ubi_ctrl  -m 4 -O 2048
mount -t ubifs -o ro,noatime  ubi0:rootfs  /rootfs

echo "Mounting rw on tmpfs"
mount    -o    rw    -t    tmpfs    aufs-tmpfs    /rw

# Remove old stuff
rm -rf /rw/tmp
rm -rf /rw/dev

#The upper is writable tmpfs /rw which should be invisible under the root dir,
#and the lower is readonly ubifs /rootfs which should be visible.
mount -t aufs -o noatime,br=/rw=rw:/rootfs=ro none /aufs


mkdir -p  /aufs/rootfs
mount --move /rootfs /aufs/rootfs


umount /sys
umount /proc
mount -t devtmpfs none /aufs/dev
exec switch_root /aufs /sbin/init
EOF

# make init executable
chmod a+x init

# create the initramfs image
find . | cpio -H newc -o > ../initramfs.cpio

#Copy the archive file to TI linux-2.6.x
cp ../initramfs.cpio /'PATH OF LINUX SOURCE'/



Patch the kernel with AUFS support

 
cd  /'Working kernel source Dir'/

# download aufs source
git clone git://git.code.sf.net/p/aufs/aufs2-standalone aufs2-standalone
cd  aufs2-standalone
git checkout origin/aufs2.2-37

#Copy aufs related files to kernel source tree
cp -a aufs2-standalone/fs .
cp aufs2-standalone/include/linux/aufs_type.h include/linux/

#Patch the kernel with AUFS related updates
patch -p1 < aufs2-standalone/aufs2-base.patch
patch -p1 < aufs2-standalone/aufs2-kbuild.patch
patch -p1 < aufs2-standalone/aufs2-standalone.patch
patch -p1 < aufs2-standalone/loopback.patch
patch -p1 < aufs2-standalone/proc_map.patch


Kernel menuconfig changes reflected in .config file.

CONFIG_INITRAMFS_SOURCE="initramfs.cpio"                                                CONFIG_INITRAMFS_ROOT_UID=0
CONFIG_INITRAMFS_ROOT_GID=0
CONFIG_INITRAMFS_COMPRESSION_NONE=y
CONFIG_AUFS_FS=y
CONFIG_AUFS_BRANCH_MAX_127=y
CONFIG_AUFS_SBILIST=y
CONFIG_AUFS_HNOTIFY=y
CONFIG_AUFS_HFSNOTIFY=y
CONFIG_AUFS_PROC_MAP=y
CONFIG_AUFS_SP_IATTR=y
CONFIG_AUFS_SHWH=y 
CONFIG_AUFS_BDEV_LOOP=y                                             
CONFIG_AUFS_DEBUG=y 
CONFIG_AUFS_MAGIC_SYSRQ=y 


Recompile the kernel with the above changes and replace the current Linux image with this new build uImage.

Changing the root file-system for package updates


mount -o remount,rw /rootfs
mount -o remount,udba=notify /
mount -o remount,mod:/rootfs=rw /
#Now make changes in the '/', finally manual copy the changes to '/rootfs/' path
#reboot the system after copy
reboot


References


Friday, April 5, 2013

Creating and Flashing UBIFs on MINI2416

UBIFS creation took me a while to understand and execute,it proved  a worthwhile experience.
my objective was to create 2 volume of UBIFS,one for rootfs and other for data.
since the uboot I was using did not support UBIFS I choosed to write from linux with the help 
of tools available with mtd-utilities.

About UBIFS on web
http://www.linux-mtd.infradead.org/faq/ubifs.html

Software used to accomplish this activity 
linux 2.6.39 configured for mtd support ported on MINI2416 with 128Mb Nand flash
precompiled tool chain and rootfs avilable on denx site
http://ftp.denx.de/pub/eldk/5.3/targets/armv5te/
toolchain 
eldk-eglibc-i686-arm-toolchain-qte-5.3.sh
rootfs with mtd-utilites compiled
core-image-minimal-mtdutils-generic-armv5te.tar.gz
install mtd-util on host ubuntu machine 
   
I created core-image-minimal-mtdutils on USB-Stick on ext2 partition,once the linux is up. 

use mtdinfo -a to list the info on mtd partitions for the board,usefull for creating ubifs image.

mtd0
Name:                           UBOOT
Type:                           nand
Eraseblock size:                131072 bytes, 128.0 KiB
Amount of eraseblocks:          8 (1048576 bytes, 1024.0 KiB)
Minimum input/output unit size: 2048 bytes
Sub-page size:                  512 bytes
OOB size:                       64 bytes
Character device major/minor:   90:0
Bad blocks are allowed:         true
Device is writable:             true

mtd1
Name:                           kernel
Type:                           nand
Eraseblock size:                131072 bytes, 128.0 KiB
Amount of eraseblocks:          24 (3145728 bytes, 3.0 MiB)
Minimum input/output unit size: 2048 bytes
Sub-page size:                  512 bytes
OOB size:                       64 bytes
Character device major/minor:   90:2
Bad blocks are allowed:         true
Device is writable:             true

mtd2
Name:                           ROOTFS
Type:                           nand
Eraseblock size:                131072 bytes, 128.0 KiB
Amount of eraseblocks:          992 (130023424 bytes, 124.0 MiB)
Minimum input/output unit size: 2048 bytes
Sub-page size:                  512 bytes
OOB size:                       64 bytes
Character device major/minor:   90:4
Bad blocks are allowed:         true
Device is writable:             true

Preparing NAND partition 
we used mtd2 partition for ubifs image.
root@generic-armv5te:~#flash_eraseall /dev/mtd2

Creating UBIFS image


mkfs.ubifs -r basicrootfs/  -m 2048 -e 126976 -c 200 -F -o basicrootfs.ubi
mkfs.ubifs -r data/  -m 2048 -e 126976 -c 792 -F -o data.ubi



To flash image to Nand from From Linux,we require to ubinize  tool on the host machine.


ubinize -vv -o basicrootfs.ubi -m 2048 -p 128KiB ./ubirootfs.cnf 






ubirootfs.cnf file captures the details of two volume created is as below.
[ubifs]
mode=ubi
image=./basicrootfs.img
vol_id=0
vol_type=dynamic
vol_name=rootfs
vol_size=20MiB
[data-volume]
mode=ubi
image=./data.img
vol_id=1
vol_type=dynamic
vol_name=data
vol_flags=autoresize








Flash the UBI file system image (ubi.img) to MTD partition "X"
ubiformat /dev/mtd2 -f basicrootfs1.ubi -s 512 -O 2048 
  
 

Changes in u-boot enviroment  to set boot arguments.
setenv bootargs 'console=ttySAC1,115200n8 noinitrd  rootwait=2 ubi.mtd=2,2048 rootfstype=ubifs root=ubi0:rootfs'


mounting the data volume
mount -t ubifs ubi0:data /media/data

Monday, September 26, 2011

Scratch Box2 , Qemu with ELDK tool chain

1 Compile and Install Qemu
a) Download qemu-0.14.1.tar.gz ,use the this link http://wiki.qemu.org/Download
b) extract the qemu, tar -xvzf qemu-0.14.1.tar.gz
c) configure Qemu source.
./configure --prefix=$HOME/qemu --target-list=arm-linux-user
d) Compile and Install the binaries.
e)make install


2 Compile and install sbox2
a) git clone git://anongit.freedesktop.org/git/sbox2
cd sbox2
b) sbox2 gives compilation error patch the source with the file given in the link below.
sbox2-patch
patch -p1 -i /PATH-TO-PATCH/sbox2.patch
c)run ./autogen.sh to create a configuration file.
d) ./configure --prefix=$HOME/sb2
e) make install
f) PATH=$PATH:$HOME/sb2/bin

Target creation.
I have a ELDK toolchain installed on /opt/eldk, so:
cd /opt/eldk/arm
sb2-init -c /opt/qemu/bin/qemu-arm armv4 /opt/eldk/usr/bin/arm-linux-gnueabi-gcc
Once sb2-init is successfully installed,we can use it for cross-compiling source.
e.g
glib cross-compiling.
Download glib-2.24.0.tar.bz2
http://ftp.gnome.org/pub/gnome/sources/glib/2.24/glib-2.24.0.tar.bz2
sb2 ./configure --prefix=/opt/gst --disable-static --with-html-dir=/tmp/dump
sb2 make install




Wednesday, July 27, 2011

GTK+ DirectFB cross compilation

GTK+, or the GIMP Toolkit, is a multi-platform toolkit for creating graphical user interfaces. Offering a complete set of widgets. GTK+ is suitable for projects ranging from small one-off tools to complete application suites.

I have download the below softwares to start with.
atk-1.30.0: http://ftp.gnome.org/pub/GNOME/sources/atk/1.30/atk-1.30.0.tar.bz2

cairo-1.8.10: http://cairographics.org/releases/cairo-1.8.10.tar.gz

DirectFB-1.4.3: http://directfb.org/downloads/Core/DirectFB-1.4/DirectFB-1.4.3.tar.gz

expat-2.0.1: http://sourceforge.net/projects/expat/

fontconfig-2.8.0: http://fontconfig.org/release/fontconfig-2.8.0.tar.gz

freetype-2.3.12: http://sourceforge.net/projects/freetype/files/

glib-2.24.0: http://ftp.gnome.org/pub/gnome/sources/glib/2.24/glib-2.24.0.tar.bz2

gtk+-2.20.1: http://ftp.gnome.org/pub/gnome/sources/gtk+/2.20/gtk+-2.20.1.tar.bz2

jpeg-8b: http://www.ijg.org/files/jpegsrc.v8b.tar.gz

tslib: http://www.zelow.no/floppyfw/download/Development/src/

libpng-1.2.39: http://sourceforge.net/projects/libpng/files/

libxml2-2.7.7: ftp://gd.tuwien.ac.at/pub/libxml/libxml2-2.7.7.tar.gz

pango-1.28.1: http://ftp.gnome.org/pub/gnome/sources/pango/1.28/pango-1.28.1.tar.bz2

pixman-0.18.2: http://cairographics.org/releases/pixman-0.18.2.tar.gz

tiff-3.9.2: http://download.osgeo.org/libtiff/tiff-3.9.2.tar.gz

zlib-1.2.5: http://zlib.net/zlib-1.2.5.tar.gz


With scratchbox2 and qemu on the host machine cross compiling was straightforward.

Cross compiling order

1) tslib
sb2 ./configure --prefix=/opt/gtkfb
sb2 make install

2) zlib
since glib require zlib
sb2 ./configure --prefix=/opt/gtkfb --shared
sb2 make install

3) glib
LDFLAGS=-L/opt/gtkfb/lib CPPFLAGS=-I/opt/gtkfb/include/ sb2 ./configure --prefix=/opt/gtkfb --disable-static --with-html-dir=/tmp/dump
sb2 make install

4) atk
export PKG_CONFIG_PATH=/opt/gtkfb/lib/pkgconfig
LDFLAGS=-L/opt/gtkfb/lib CPPFLAGS=-I/opt/gtkfb/include/ sb2 ./configure --prefix=/opt/gtkfb --disable-glibtest
sb2 make install

5) jpeg-8b
sb2 ./configure --prefix=/opt/gtkfb --enable-shared --enable-static
sb2 make install

6) libpng
LDFLAGS=-L/opt/gtkfb/lib CPPFLAGS=-I/opt/gtkfb/include/ sb2 ./configure --prefix=/opt/gtkfb
sb2 make install

7) expat-2.0
sb2 ./configure --prefix=/opt/gtkfb
sb2 make install

8) freetype
sb2 ./configure --prefix=/opt/gtkfb
sb2 make install

9)libxml
LDFLAGS=-L/opt/arm/gst/lib CPPFLAGS=-I/opt/arm/gst/include/ sb2 ./configure --prefix=/opt/gtkfb --without-python
sb2 make install

10)freetype-2.3.12
LDFLAGS=-L/opt/arm/gst/lib CPPFLAGS=-I/opt/arm/gst/include/ sb2 ./configure --prefix=/opt/gtkfb --with-freetype-config=/opt/gtkfb/bin/freetype-config

11) tiff -- not used gcc 4.3.2
LDFLAGS=-L/opt/arm/gst/lib CPPFLAGS=-I/opt/arm/gst/include/ sb2 ./configure --prefix=/opt/gtkfb --enable-shared
sb2 make install

12) DirectFB
LDFLAGS=-L/opt/arm/gst/lib CPPFLAGS=-I/opt/arm/gst/include/ sb2 ./configure --prefix=/opt/gtkfb --with-gfxdrivers=none --with-inputdrivers=all --enable-png --enable-jpeg --enable-tiff=no --enable-zlib --enable-sdl=no --enable-gif=no --disable-x11
sb2 make install

13) pixman-0.18.2
LDFLAGS=-L/opt/arm/gst/lib CPPFLAGS=-I/opt/arm/gst/include/ sb2 ./configure --prefix=/opt/gtkfb --disable-gtk

14) cairo
LDFLAGS=-L/opt/gtkfb/lib CPPFLAGS=-I/opt/gtkfb/include/ sb2 ./configure --prefix=/opt/gtkfb --without-x --disable-xlib --disable-xlib-xrender --enable-directfb --enable-freetype --disable-win32 --enable-pdf --enable-ps --enable-svg --enable-png

15) Pango
LDFLAGS=-L/opt/gtkfb/lib CPPFLAGS=-I/opt/gtkfb/include/ sb2 ./configure --prefix=/opt/gtkfb --without-x

16 gtk
LDFLAGS=-L/opt/gtkfb/lib CPPFLAGS=-I/opt/gtkfb/include/ sb2 ./configure --prefix=/opt/gtkfb --with-gdktarget=directfb --without-x --without-libtiff --disable-glibtest --disable-cups

Now to compile a simple helloworld.c pro-gramme with these libraries.
export PKG_CONFIG_PATH=/opt/gtkfb/lib/pkgconfig
arm-linux-gcc -Wall -g helloworld.c -o helloworld `pkg-config --cflags gtk+-2.0` `pkg-config --libs gtk+-2.0` -Wl,-O1 -Wl,-rpath,/opt/gtkfb/lib

Monday, May 2, 2011

mini2440 as USB mass storage gadget device

To make mini2440 as USB mass storage device,I needed to check both the schematics and source code. I am not sure the reason GPG12 GPIO port pin is used by touch screen, however this is not captured in schematics. also referred http://www.linux-usb.org/gadget/file_storage.html link to establish this task.

Step 1:
so the first step is to disable the touch screen driver in the kernel source tree.

step 2:
--- USB support
[*] USB device filesystem
<*> USB Gadget Support --->
File-backed Storage Gadget

step 3:
cross compile the kernel,boot the new kernel and filesystem. go to compiled linux source tree and download g_file_storage.ko to mini2440.

step 4:
Creating a backing storage file
since the mini2440 has 64 MB flash, I allocated 10MB space for backup storage file.
bash# dd bs=1M count=10 if=/dev/zero of=/root/backing_file

step 5:

Partitioning the backing storage

     bash# fdisk /root/backing_file
Device contains neither a valid DOS partition table, nor Sun or SGI disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

You must set heads sectors and cylinders.
You can do this from the extra functions menu.

Command (m for help):

Give the "x" (eXpert or eXtra) command:

Command (m for help): x
now select Heads, Sectors, and Cylinders
g_file_storage uses a sector size of 512 bytes, so 8 sectors/track will give us 4096 bytes
per track
Expert command (m for help): s
Number of sectors (1-63): 8
Warning: setting sector offset for DOS compatiblity

With 4 KB per track, 16 heads will give us a total of 64 KB per cylinder
Expert command (m for help): h
Number of heads (1-256): 16

Since we've got 64 KB per cylinder and 10 MB total, we need to use 320
cylinders.
Expert command (m for help): c
Number of cylinders (1-131071): 320

Now return to the normal menu (the "r" command):
Expert command (m for help): r

creating a primary partition

     Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1024, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1024, default 1024):
Using default value 1024

Since you want to use the gadget with a Windows host, you should change the
partition type (the "t" command) to FAT32 (code "b")
Command (m for help): t
Partition number (1-4): 1
Hex code (type L to list codes): b
Changed system type of partition 1 to b (Win95 FAT32)

Finally write out ("w") the partition table to the backing storage:
Command (m for help): w
The partition table has been altered!

Adding a filesystem
   You use the losetup program to set up the loop
device driver with the proper offset:

# losetup -o 4096 /dev/loop0 /root/backing_file

and then you can mount it:

# mount -t vfat /dev/loop0 /mnt/loop

/dev/loop0
is mapped to the partition within the backing
storage. You can create a filesystem on it:
# mkdosfs /dev/loop0


now do "insmod
g_file_storage file=/root/backing_file" on mini2440,then connect mini2440 to host PC.
the FAT32 partition with will be visible as storage device on Host system

Wednesday, March 31, 2010

ELDK on mini2440

The Embedded Linux Development Kit (ELDK) includes the GNU cross development tools, such as the compilers, binutils, gdb, etc.support different Target architecture including arm.

Download link
http://www.denx.de/wiki/DULG/ELDKAvailability


Creating enviroment on host PC.
linuxhost > mkdir eldk
linuxhost > mkdir install_mount
linuxhost > cd eldk/
linuxhost/eldk > su
linuxhost/eldk # mount -o loop,exec ../arm-2007-01-21.iso ../install_mount/
linuxhost/eldk # exit


linuxhost/eldk > ls ../install_mount/
. arm ELDK_FIXOWNER etc install RPMS tools
.. common ELDK_MAKEDEV icons README.html sys version


linuxhost/eldk > ../install_mount/install arm
linuxhost/eldk > chmod u+x eldk_init

now unmount the ISO image as root

linuxhost/eldk > ./eldk_init arm
export these variable to environment,or include them as part of .bashrc file.
ARCH=arm
CROSS_COMPILE=arm-linux-
DEPMOD=/opt/eldk/usr/bin/depmod.pl
PATH=/opt/eldk/usr/bin:/opt/eldk/bin


Using ELDK for mini2440

compiling the Kernel...
Error found while compiling the kernel
kernel/bounds.c:1: error: invalid ABI option: -mabi=aapcs-linux
Fix is to un select Kernel Features ---> [ ] Use the ARM EABI to compile the kernel
Now do make in kernel source,the kernel is successfully build on ELDK.

Building the Root file system
untar the root_qtopia.tar.gz which is available on CD.
cd root_qtopia
rm bin sbin lib usr -rf
cp /opt/ELDK/arm/lib . rf

BusyBox combines tiny versions of many common UNIX utilities into a single small executable,I have used busybox-1.16.0 for my filesystem.
compilation is very straightforward. make menuconfig,
In BusyBox Settings -->Build Options , use 'arm-linux-' as cross compiler prefix.
In Installation Options-> , update the BusyBox Installation prefix
busybos build will create 'bin','sbin','usr' folders in its installation folder.
copy these folders to root_qtopia/
cp /{bin,sbin,usr} root_qtopia/ -rf

now yaffs2 image for root_qtopia and upload it to mini2440 board.

mkyaffs2image root_qtopia rootfs.img