A Bit of Technical Stuff

New Blog

2010-10-17T04:06:00.000-07:00

This blog is moved to my new website
http://neependra.net/

Jiffy, Jiffies & HZ

2008-09-24T03:35:00.000-07:00

Jiffies : The jiffies variable holds the number of times the system timer popped since the system booted.

HZ: The number of timer ticks per second, is contained in the kernel variable HZ.
The kernel increments jiffies, HZ times every second

Jiffy: On a kernel with a HZ value of 100, a “jiffy” is a 10-millisecond duration, while on a kernel with HZ set to 1000, a jiffy is only 1-millisecond.

For more info
http://www.linux-mag.com/id/2272 (The Passage of Time)

How do log every thing printed on the terminal

2008-09-17T00:29:00.000-07:00

Sometimes we just keep working on a terminal and get lost what all we did.
Isn't it good if we can get some kind of logs which can give us all steps we followed.

First thing which comes in mind is the "history" command, but that will not give the standard outputs which we got after running the commands.

So, here is the solution.Use a command called "script".

e.g.

$ script standard_out.txt
Script started, file is standard_out.txt
$ date
Mon Sep 8 07:57:20 IST 2008
$ asdf
bash: asdf: command not found
$ exit
Script done, file is standard_out.txt

To exit you need to press "ctrl+D".

Now lets check the content of "standard_out.txt" file.

$ cat standard_out.txt
Script started on Mon 08 Sep 2008 07:57:10 AM IST
$ date
Mon Sep 8 07:57:20 IST 2008
$ asdf
bash: asdf: command not found
$ exit
Script done on Mon 08 Sep 2008 07:57:38 AM IST

For more information look at the man page of ''script"'.

Deleting a speical character file

2008-09-03T22:45:00.000-07:00

Sometimes we need to delete a special character file, which we can not delete with normal rm command.So here is the solution...delete it with the help inode number.

$find . -inum __inode number from ls -i of the file __ -exec rm {} \;

Highlighting the searched keyword with 'grep'.

2008-08-11T08:06:00.000-07:00

Add following lines in the ".bashrc" file, which is in your home directory.

alias grep="grep --color"

export GREP_COLOR="1;32"

You can run either run following command to take this change in-effect

$ source ~/.bashrc

Logout and then login.

Global File System(GFS) -- Part1

2008-07-11T02:59:00.001-07:00

File system is a method for storing and organizing computer files and the data they contain to make it easy to find and access them.
There are different types of file system available
* Disk file systems (EXT2, EXT3,XFS,FAT,NTFS,GFS,OCFS)
* Flash file systems
* Network file systems(NFS)
* Special purpose file systems(proc file system)

Each file system has it own structure.So when different file systems are mounted on the same system, how the user has uniform view.
For example mount different file systems under "/mnt" and then run "ls -lR /mnt".What happens?

We get the listing of the files, irrespective of the the file-systems.

VFS (Virtual File System) acts as a bridge between the applications and
the real file systems.VFS describes the system's files in terms of superblocks and inodes in much the same way as the EXT2 file system uses superblocks and inodes.

When each file system is initialized, it registers itself with the VFS.When user /application wants to access any file, it request the VFS and then VFS
calls the file system specific functions/routines to access the requested file.

GFS file system is required when more than one system needs simultaneous access to the shared storage.With files system like EXT3 you can not do this.
Try following:-
- Get the shared storage (From SAN/ iSCSI) and assign is multiple systems
- From one system create EXT3 file system on it.
- Mount the storage to all the system.
- Try to create files on that mount point from one system.

Are the newly created are visible from other system ?
We need to remount the storage on other systems to get a consistent view.

With GFS, clients modify the shared data by accessing it atomically.Before accesing the data client acquires the lock for it,modifies the data and then releases the lock.Clients access storage devices like processors of a shared memory multiprocessor computer (SMP) access memory.Each of the GFS client thinks that it has the full access to the storage and they are unaware of each other.

DLM (Distributed Lock Manager), GULM(Grand Unified Locking Manager), and Nolock are locking protocols which GFS uses.
With RHEL3/4, you can use both DLM/GULM. With RHEL5 you can only use DLM.

Nolock is used when you want to use GFS on a stand alone system.Now why would someone like to do that? GFS allocated inodes dynamically. Each file takes one inode, so if someone has to create too many small files he/she can use GFS. In some file systems inode number are limited.

The network storage pool (NSP) provides each machine with a unified storage address space. A device driver layered on top of SCSI and Fibre Channel drivers implements the NSP. The NSP driver translates from the logical address space of the file system to the address space of each device.Subpools divide NSPs into groups of similar device types. These subpools inherit characteristics from underlying devices and network connections.

GFS organizes file systems into several resource groups (RG). Resource groups distribute file system resources across the entire NSP; multiple resource groups can exist per device. Resource groups are essentially mini-file systems. Each group possesses an information block, data bitmaps, dinodes,and data blocks. Resource group blocks contain information similar to traditional superblocks. In normal operation, resource groups are transparent to users.

References:
http://www.diku.dk/undervisning/2003e/314/papers/soltis97global.pdf
http://en.wikipedia.org/wiki/Global_File_System
http://www.science.unitn.it/~fiorella/guidelinux/tlk/node102.html#SECTION001120000000000000000

Jabalpur LUG event, 17th June'08

2008-06-23T19:24:00.000-07:00

With the help of jbp-lug members I have organized a LUG event at Takshila Engineering college, Jabalpur.
http://tech.groups.yahoo.com/group/jbp-lug/

Meeting Minutes:
http://tech.groups.yahoo.com/group/jbp-lug/message/393
Pics:
http://tech.ph.groups.yahoo.com/group/jbp-lug/photos/browse/d30d

Around 20 students from different engineering colleges attended the event. 80% students of city like Jabalpur lack the practical understanding of the subjects; they just somehow read them to pass the exams. As per my understanding lack of resources, awareness and mentors is the root cause of this.Groups like LUGs can be very useful here.

GEEP session on Virtualization

2008-05-27T06:03:00.000-07:00

GEEP --- GEEKs Of Pune...
GEEP is a non-profit group to promote Linux system software.You can find more details at :-
http://tech.groups.yahoo.com/group/geep_linux/

On 25th May'08 GEEP has organized the session on Virtualization.There were two presentations:-
a. For beginners by me.
b. For advance users by Mr. Amit Shah

If we put "vitualization" on any search engine we get 'n' number of results and get confused.
So, I put more emphasis on the basics.

I started my talk by comparing abstraction and virtualizarion.In virtualization we do not hide the underlying details and real system is transformed so that it appears to be different. For abstraction you can think of the OSI model.

Then I talked about the "Why Virtualization". Some of the reasons are:-
* Server consolidation *Multiple execution environments
* Virtual hardware * Debugging
* Software migration (Mobility) * Testing/Quality Assurance
* Live Migration * Balancing Resources

If we talk about virtualization then we definitely has to talk about the computer architecture.

* ISA (Instruction set architecture ) divides hardware and software. There are two kind of ISA, User ISA and System ISA.

* ABI (Application Binary Interface) provides a program with access to the hardware resource and services available in a system. It consists of User ISA and System Call Interfaces

* API (Application Programming Interface) is a source code interface that an operating system, library or service provides to support requests made by computer programs.
clib in Unix environment : supports the UNIX/C programming .

By taking above things in mind we can divide Virtual Machines in two categories:-

Process VM at the ABI level: Virtualization of individual processes.
E.g., running x86 applications on Alpha CPU.
Java is another example of process virtualizaton. When we compile a java program, it is compiled into some kind a pseudo language which can interpreted by a jvm(java virtual machine). That is java is platform independent. Please note that jvm is different is each platform.

Process virtualization is a matter of correct functioning and not a matter of performance.

System VM at the ISA level.
Virtualization of complete systems
E.g., running Linux (and its applications) on Windows.

In system virtualiztion I talked about hardware emulation, hypervisor, full and para virtualization.I referred following for the same.
http://www.ibm.com/developerworks/library/l-linuxvirt/index.html

Then I talked about a bit kvm and handed over to Amit.
Amit's presentation was awesome.

I was surprised to know that kvm is now supporting para-virtualization as well.

Why Para-vitualization was needed?

2008-05-18T06:42:00.000-07:00

Why Para-vitualization was needed?
I had this question in my mind for some time but the answer was not clear to me. So, I read some docs and figured it out.

From http://en.wikipedia.org/wiki/Intel-VT
x86 virtualization is the method by which x86-based "guest" operating systems are run under another "host" x86 operating system, with little or no modification of the guest OS. The x86 processor architecture did not originally meet the Popek and Goldberg virtualization requirements. As a result, it was very difficult to implement a general virtual machine on an x86 processor.

From http://www.kernelthread.com/publications/virtualization/
One of the most popular architectures, IA-32(x86), is not virtualization friendly. The analysis in a paper titled Analysis of the Intel Pentium's Ability to Support a Secure Virtual Machine Monitor reports at least seventeen instructions on the Pentium that make it "non-virtualizable". IA-32's privileged instructions cause a General Protection Exception when executed in non-privileged mode. Instructions like STR can be problematic: STR can be executed at any privilege level, but it tells you the security state of the machine (the value it retrieves has the Requestor Privilege Level, or RPL).

To overcome this guest OS was modified, so that it was aware that it is being virtualized.
And this is called para-vitualization.

From http://www.ibm.com/developerworks/linux/library/l-linux-kvm/
Older x86 processors produce different results for certain instructions depending upon the domain of execution. This creates a problem, because the hypervisor should execute in only the most protected domain. For this reason, virtualization solutions such as VMware will pre-scan code that is to execute to replace these instructions with trap instructions so that the hypervisor can handle them appropriately. Xen, supporting a cooperative method of virtualization, requires no modification, because the guest is aware that it's being virtualized and is modified. KVM simply ignores this problem and mandates that if you want virtualization, you'll have to run on newer hardware.