Wednesday, December 31, 2014

Why we may want to disable Nagle's algorithm

Nagle's algorithm is a TCP optimization in the kernel stack that waits to aggregate small chunks of bytes before sending packets on a TCP connection. This approach optimizes the amount of frame overhead spent in sending very small packets over the network. However, when the data is fairly sporadic, this could also lead to an increase in the average delay experienced.

This can be done by:
echo 1 > /proc/sys/net/ipv4/tcp_low_latency

Thursday, December 11, 2014

When to use GFP_ATOMIC over GFP_KERNEL

Using the GFP_ATOMIC option with kmalloc instructs the memory allocator to never block waiting for a page. So the execution of the instruction will happen instantaneously - either with the allocation of the desired memory or not. Because the kernel does not have a chance to wait and free
up memory or reclaim memory for new allocations, GFP_ATOMIC has a higher chance of failing than GFP_KERNEL.

In order to make sure that the memory can be DMA’d, bitwise or either GFP_KERNEL or GFP_ATOMIC with the GFP_DMA flag as shown below:
1 . . .
2 buf = kmalloc (BUF_LEN, GFP_DMA | GFP_KERNEL) ;
3 i f ( ! buf )
4 e r r o r handling
5 . . .

Wednesday, December 10, 2014

Why is the buddy system needed in Linux

The buddy system is a mechanism for page management in Linux. It is needed to make sure that the free memory does not get fragmented and unusable. An alternative to the buddy system would be to use the memory management unit (MMU) support to rewire or re-arrange blobs of free pages together to construct larger contiguous pages. However, this will not work for DMA systems which bypass the MMU. Also, modifying the virtual address on a continual basis would make the paging process slow.

Debugging on the buddy system can be done by printing the current stats. This is supported under the /proc/buddyinfo file. As described in the guide from centos.org, fragmentation issues can be debugged. A sample output from the same site is as shown below:
cat /proc/buddyinfo

Monday, December 8, 2014

Customizing latex footnotes to alphabets

There is a simple hack to do this:
\renewcommand{\thefootnote}{\arabic{footnote}}Arabic numerals, e.g., 1, 2, 3...
\renewcommand{\thefootnote}{\roman{footnote}}Roman numerals (lowercase), e.g., i, ii,... 
\renewcommand{\thefootnote}{\Roman{footnote}}Roman numerals (uppercase), e.g., I, II, ... 
\renewcommand{\thefootnote}{\alph{footnote}}Alphabetic (lowercase), e.g., a, b... 
\renewcommand{\thefootnote}{\Alph{footnote}}Alphabetic (uppercase), e.g., A, ... 
\renewcommand{\thefootnote}{\fnsymbol{footnote}}A sequence of nine symbols,
Changed mine to alphabets. Looks much neater now.

Thursday, October 9, 2014

Difference between Inkscape and GIMP

Inkscape is great for vector graphics i.e. logo creation, or any other vector creation.

GIMP is awesome for raster graphics i.e. photo editing etc.

Tuesday, September 30, 2014

Why not use C++ instead of C for ...

This question pops up again and again, either for operating systems or databases or something else.

Primarily this has to deal with: How Linus Torvalds feels about C++

This is what happened on one of the the questions concerning the use of C++ over C for some GIT thread..

Enjoy!

--
From: Linus Torvalds <torvalds <at> linux-foundation.org>
Subject: Re: [RFC] Convert builin-mailinfo.c to use The Better String Library.
Newsgroups: gmane.comp.version-control.git
Date: 2007-09-06 17:50:28 GMT (2 years, 14 weeks, 16 hours and 36 minutes ago)

On Wed, 5 Sep 2007, Dmitry Kakurin wrote:
>
> When I first looked at Git source code two things struck me as odd:
> 1. Pure C as opposed to C++. No idea why. Please don't talk about portability,
> it's BS.

*YOU* are full of bullshit.

C++ is a horrible language. It's made more horrible by the fact that a lot
of substandard programmers use it, to the point where it's much much
easier to generate total and utter crap with it. Quite frankly, even if
the choice of C were to do *nothing* but keep the C++ programmers out,
that in itself would be a huge reason to use C.

In other words: the choice of C is the only sane choice. I know Miles
Bader jokingly said "to piss you off", but it's actually true. I've come
to the conclusion that any programmer that would prefer the project to be
in C++ over C is likely a programmer that I really *would* prefer to piss
off, so that he doesn't come and screw up any project I'm involved with.

Monday, September 29, 2014

Solution: Unknown symbol “__aeabi_ldivmod”

You might notice that while compiling for your 32bit platforms your kernel module compiles. However, when you are inserting it we see a failure (either at boot or while explicitly doing an insmod or modprobe).

The reason this "Unknown symbol in module" is seen is because you are in some instruction trying to do a 64bit division in the Linux kernel for an ARM platform (32bit).

Why is the symbol missing though if everything compiles. The compiler wants to do the 64bit div with slow library functions which Linux does not implement. So when the code is run, the symbol (__aeabi_ldivmod) is not found.

The solution to this problem is to use do_div() while including <asm/div64.h>.

do_div has its own side effects and if you search enough you will find that there are multiple cases where "do_div is considered harmful". But for now, if you are not doing this division frequently, it might be a good idea to just use do_div() and get your work done.

From div64.h:

// the semantics of do_div() macros are:
uint32_t do_div(uint64_t *n, uint32_t base) {
  uint32_t remainder = *n % base;
  *n = *n / base;
  return remainder;
}

Simple example:

void  _do_64bit_div(int64_t *operand_ptr, int64_t base)
{
#ifndef  FULL64BITSUPPORT
    // do_div for older platforms.
    uint64_t operand, remainder;
    operand = *operand_ptr;
    remainder = do_div(operand, base);
    *operand_ptr = operand;
#else
    // Direct div for newer platforms.
    unit64_t result;
    *operand_ptr = *operand_ptr / base;
#endif
}

Just call this function with the parameters (&operand / operator) and get the result in operand.


Sunday, September 14, 2014

Engineering student's list of essential linux utilities

I will keep adding to this list as I add  more software to the list. This is what I have installed as of now.

Research software:

  1. GNU octave - cant do without this one (package - ubuntu)
  2. gnuplot - plotting essentials - (package - ubuntu)
  3. NS2 simulator - (package - ubuntu)

General utilities
  1. vim (package - ubuntu)
  2. tmux - terminal emulator (package - ubuntu)
  3. keepassx - cross platform key manager (package - ubuntu)
  4. filesync - open source sync tool (package - ubuntu)
  5. Comix - CBR reader for your comic book readings :)
  6. Xscreensaver with the glsideshow package - for creating a digital picture frame of your favorite memories when your linux machine is not working its butt off!

Productivity packages:
  1. Inkscape for generating phenomenal scalar and raster images. Removed GIMP from the list.
  2. Texworks latex suite - super latex productivity
  3. SourceTree based free git support for one tree.
  4. Bash completion package - for command line completions

Please note I am not endorsing any of these apps. These are just the things that I found useful.

Wednesday, September 10, 2014

How to get right click key on a windows keyboard

Short answer use shift + F10

Long answer:
My keyboard did not have a direct F10 key (it was FN+F10) so I had to make one.
You can remap key settings using a nice GNU freeware: keytweak-setup

I used it to remap my prtscr key to F10 and then I can just use shift + PRTscr to do a right click.


Wednesday, August 20, 2014

Kernel crash debugging: BUG: scheduling while atomic

Why do you see that print
"Scheduling while atomic" indicates that you've tried to sleep somewhere that you shouldn't - like within a spinlock-protected critical section or an interrupt handler.

Things to check:

1. In this case you should check if you are actually returning from some code that could cause the lock not to be released or actually sleeping in some part of the code.

2. Another error that may be spitted out during such a crash is :
BUG: workqueue leaked lock or atomic
This clearly indicates that you were not unlocking a certain lock, which could be typically caused by returning from a routine before the lock is released.

3. You are running something under a spinlock and it could not run to completion for some reason causing the kernel scheduler to be eventually invoked (schedule() ) with the spinlock still held.

These are few of the most popular reasons why you might see this print. If your kernel is configured to crash under such conditions, try and take a look at the backtrace which might hint towards what was running when this crash happened.

Friday, July 11, 2014

Poking the ethernet driver with the ethtool

1. Get basic information about the interface
[mylinuxbox@mylinuxbox-linux ~]$ ethtool -i eth4
driver: e1000e
version: 2.1.4-k
firmware-version: 0.13-4
bus-info: 0000:00:19.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no


2. Dump all the hardware registers
[root@mylinuxbox-linux mylinuxbox]# ethtool -d eth4
MAC Registers
-------------
0x00000: CTRL (Device control register)  0x18100240
      Endian mode (buffers):             little
      Link reset:                        normal
      Set link up:                       1
      Invert Loss-Of-Signal:             no
      Receive flow control:              enabled
      Transmit flow control:             enabled
      VLAN mode:                         disabled
      Auto speed detect:                 disabled
      Speed select:                      1000Mb/s
      Force speed:                       no
      Force duplex:                      no
0x00008: STATUS (Device status register) 0x00080083
      Duplex:                            full
      Link up:                           link config
      TBI mode:                          disabled
      Link speed:                        1000Mb/s
      Bus type:                          PCI
      Bus speed:                         33MHz
      Bus width:                         32-bit
0x00100: RCTL (Receive control register) 0x04008002
      Receiver:                          enabled
      Store bad packets:                 disabled
      Unicast promiscuous:               disabled
      Multicast promiscuous:             disabled
      Long packet:                       disabled
      Descriptor minimum threshold size: 1/2
      Broadcast accept mode:             accept
      VLAN filter:                       disabled
      Canonical form indicator:          disabled
      Discard pause frames:              filtered
      Pass MAC control frames:           don't pass
      Receive buffer size:               2048
0x02808: RDLEN (Receive desc length)     0x00001000
0x02810: RDH   (Receive desc head)       0x00000051
0x02818: RDT   (Receive desc tail)       0x00000040
0x02820: RDTR  (Receive delay timer)     0x00000000
0x00400: TCTL (Transmit ctrl register)   0x3003F0FA
      Transmitter:                       enabled
      Pad short packets:                 enabled
      Software XOFF Transmission:        disabled
      Re-transmit on late collision:     disabled
0x03808: TDLEN (Transmit desc length)    0x00001000
0x03810: TDH   (Transmit desc head)      0x0000007A
0x03818: TDT   (Transmit desc tail)      0x0000007A
0x03820: TIDV  (Transmit delay timer)    0x00000008
PHY type:                                unknown

Wednesday, July 9, 2014

Simple technique to implement moving averages in Java, C or C++

The oneline solution is:
accumulator = (alpha * new_value) + (1.0 - alpha) * accumulator
Here,

Accummulator - holds the value being tracked
alpha - value between 0 and 1.

The more aggressive the alpha (closer to 1) the  faster the moving average adapts to the recent values. This is an exponential moving average.

Cheatsheet: Convert jiffies to msecs quickly

jiffies / HZ          /* jiffies to seconds */
jiffies * 1000 / HZ   /* jiffies to milliseconds */

Lesser important ones:
2*HZ     /* 2 seconds in jiffies */
HZ       /* 1 second in jiffies */
foo * HZ /* foo seconds in jiffies */
HZ/10    /* 100 milliseconds in jiffies */
HZ/100   /* 10 milliseconds in jiffies */
bar*HZ/1000 /* bar milliseconds in jiffies */

Wednesday, July 2, 2014

Solved: Apple MACBook does not reconnect to WiFi

A simple solution to this problem is to disable and enable your wifi connection. This can be done over a SSH session to the computer or at command line in the terminal utility with the following commands.

Turn off wifi on your macbook from the Mac OSX terminal command line:

networksetup -setairportpower en0 off

Turn on wifi on your macbook from the Mac OSX terminal command line:

networksetup -setairportpower en0 on

Tuesday, January 28, 2014

Easy solution: vim caught deadly signal segv vim preserving files vim finished

Ran into this irritating problem when everything was initially working fine on my system.

I had copy pasted something into the command window by mistake which caused vi to crash and keep spitting this out. I am using a Fedora version.

Thing that I tried which did not work:
1. Rebooting
2. Uninstalling and reinstalling vi

Solution: Under your home directory there is a file .viminfo which contains all the cached vi information. Delete this file. It will get recreated afresh the  next time vi starts. Just deleting this file fixed the problem for me.

Tuesday, January 7, 2014

Difference between low resolution and high resolution kernel timers

Low resolution timers
Low resolution timers in the kernel are implemented through the standard API:
* init_timer()
* add_timer()
* mod_timer()

These timers are called low resolution because they measure their time units in kernel jiffies.
All the logic for the low resolution timers is supported by <linux/timer.h>

Structure used to support the timers is struct timer_list { }.

High resolution timers
The new high resolution timers approach to kernel timers uses time units in nano-seconds. This consists of a very small list of timers maintained on a CPU which is sorted as a red-black tree. Though the insertion and deletion to this list is not as fast as that of the low res timers (as per the intended purpose), the list itself is small.  All the code needed for the high resolution timers is supported by <linux/ktime.h>

Time measured in nanoseconds is held by ktime_t. The structure used for hrtimers, is struct hrtimer {}.

Wednesday, January 1, 2014

Difference between priority inversion and priority inheritance

Consider two processes A and B which use a common lock. Process A has a much higher priority than process B. In this case if process B holds the common lock, then process A will not have a chance to run until process B releases the lock. In the mean time, if the scheduler runs it could always pick a process C which has priority in between A and B to run.

Hence, now process C which has priority less than A runs before A thereby making the priorities assigned to the processes useless. Also, since C has a higher priority than B it runs longer further preventing B from releasing the common lock for A. In the worst case we can imagine a lot of other processes like B which have priorities between A and B.

Why Pre-emption does not work here:
In this case even if we enable kernel pre-emption, all it can ensure is making the process B relinquish control. However, process A still can't run until the common lock is released by B.

Solutions:

1. Careful and minimal use of locks: Works best in most cases.

2. Priority inheritance: In this mechanism the process holding the lock inherits the priority of the highest priority process holding the lock. In our example, this will ensure that B has the same priority as A and runs often enough to release the resource soon enough for A to use it. This has been an often contended approach to handling priority inversion and is not the best way out of the situation.