» Create turbocharged storage using tmpfs

Everyone knows that RAM is so much faster than a hard disk. To illustrate, while a current SATA disk has peak transfer rates of 375 MB/s, current RAM can do a mind blowing 12,500 MB/s! Normally only the system itself makes use of this ultra fast storage, but we can also access this space directly. And that opens a great window of opportunity.

Possible uses

There is an unlimited number of uses for this technology, but here are 3 from my own experience:

  • What if you're running a blog, so successful that your server can't handle it. Although your blogging software caches plain html files in the /cache dir to speed up processing, it still doesn't give you enough performance.
  • What if you're an internet host and you want to show off your massive bandwidth by having users download a dummy .bin file of 100MB. You'll find that if many users access this file at the same time, your hard drive becomes the slow factor and you're running into disk IO problems.
  • What if you're running a PXE server with an ISO stored on it, and an entire webcluster is accessing this file for installation. Again. Your hard drive will not be able to cope with these kinds of speeds.

If only you could store these files in memory..

How does it work?

Everybody who's running a linux server must have seen the /dev/shm on their system.

This not a normal directory on your machine. It is intended to appear as a mounted file system, but one which uses virtual memory instead of a persistent storage device. The standard /dev/shm grows automatically as more space is needed, but is by default limited to half of your physical RAM. If you have 2GB, it can grow to 1GB at most.

So everything you copy to that place is in fact stored in your RAM. And that's cool because your RAM is about 33 times faster than your normal filesystem!

Let's do this

In this article I presume you have basic knowledge of server administration, be carefull because you could really mess things up if you've got no clue what you're doing. I warned you!

Use current volume

As told before, you probably already have a /dev/shm on your linux system. So just copy a file to it:

cp -af /root/100mb.bin /dev/shm/

And now it's in your RAM! Using this file will be 30 times faster than before!

Enlarge current volume

But maybe the limit of half of your physical RAM just does not cut it for you. Then you might want to increase the maximum size of this volume to 4GB:

mount -o remount,size=4G /dev/shm

Create a new volume

Another possibility is to create a brand new memory device. We can do this with the filesystem type: tmpfs. Let's say you want to create a tmpfs instance on /var/www/www.mysite.com/ramdrive that can allocate a max of 500MB RAM and that can only be changed by root, but accessed by everyone (like Apache):

mkdir -p /var/www/www.mysite.com/ramdrive
mount -t tmpfs -o size=500M,mode=0744 tmpfs /var/www/www.mysite.com/ramdrive

Restore that volume everytime your server boots

Easy, just make sure the directory exists, and add the following line to your /etc/fstab :

tmpfs /var/www/www.mysite.com/ramdrive tmpfs size=500M,mode=0777 0 0

This will create a ramdrive of max 500MB in /var/www/www.mysite.com/ramdrive everytime your server boots. The mode 0777 will give it full access for everybody on your system, so just change that to a suitable umask.

Done.

Pitfalls

Now as with everything too cool, there is a pitfall:

  • Everything in tmpfs is temporary in the sense that no files will be created on your hard drive. If you reboot, everything in tmpfs will be lost.
    So you will need to create a script that automatically restores the files or applications that you need. You can let this script run at boot time, or schedule it as a cronjob.

Stay up to date

You can track my blog rss articles and rss comments. You may also find my rss bookmarks interesting. Or twitter Follow me on Twitter


Like this article?

   Then Digg it!
Or use another bookmark button below to show your support &
help me spread the word.


tags: devshm, tmpfs, disk IO, linux, ubuntu, RAM, PXE, SATA, apache, performance, umask
category: Howto - System
read: 27,951 times

Add comment

(required, shown)(required, not shown)for syntax highlighting

[CODE="Javascript"]
your_code_here();
[/CODE]

Replace "Javascript"
with "php", "text", etc.
code (to make sure you are not a spammer)

 Track replies: rss feed comments feed

Comments

#12. Ryan on 10 August 2008

Default avatar:Ryan@ sciurus

yes its true that most files will be cached but... if say you have a mail server with thousands of small files you can force all the files into ram so that when the file is called for its in ram and ready... cached files must be read first before they can get into ram

so if the file only gets read once and never again (untill its pushed out of cache) the cache was never usefull. a tmpfs mount isnt ment for systemwide use but for when files need fast access times or very high thoughputs

#11. Kevin on 17 July 2008

Member avatar: Kevin@ sciurus: Thanks for the thorough input. I couldn't tell you if it's because the usual cache is purged to often, not enough files are cached, or maybe it's the first read that frustrates performance; but storing cached files on tmpfs greatly improved this sites access times.

#10. sciurus on 27 June 2008

Default avatar:sciurusA tmpfs is useful in some circumstances, but not for the three examples you cited. In those cases, once the file is read once it will be cached in memory. There's no need for a tmpfs to accomplish this. The cached line in /proc/meminfo tells you how much of a system's memory is being used for cache; and you can use the program fincore to see how much of a particular file is cached. It's easy to make a demonstration with dd. For example,

With a cold cache:
dd if=sda2.img of=/dev/null
2230345+1 records in
... [more] 2230345+1 records out
1141936658 bytes (1.1 GB) copied, 28.0823 s, 40.7 MB/s

Now with the hot cache:
dd if=sda2.img of=/dev/null
2230345+1 records in
2230345+1 records out
1141936658 bytes (1.1 GB) copied, 1.75497 s, 651 MB/s

So unless there's a lot of disk activity on a system thats causing the files you care about reading to be flushed from cache, don't bother with a tmpfs.

#9. Guru on 08 January 2008

Default avatar:GuruThis is really cool feature of unix ..esp for non-unix guy :-)...and thanks to Kevin to make it so simple

#8. Sanjay More on 23 November 2007

Default avatar:Sanjay MoreCool..
I used this technique to fire my thousands for daily newswires using postfix... gr8 help indeed..
Thanks to Matt as well for his post
http://archives.neohapsis.com/archives/postfix/2004-11/1799.html

#7. Kevin on 23 September 2007

Member avatar: Kevin@ Rick: yes it's as easy as that.

#6. Rick on 22 September 2007

Default avatar:Ricknow do I move, at startup, a frequently used file, say temdb to the ramdrive and use it. I dont mind if I loose it... thet is the Idea... or let it Linux use cahe?.. what is better?

#5. fotzenkaiser on 09 September 2007

Default avatar:fotzenkaisernice read

#4. Kevin on 14 August 2007

Member avatar: Kevin@ Joseffb: disk IO is not really a bottleneck for gaming is it? Almost every important aspect of a game is loaded in memory anyway sooner or later. The only thing I can think of that you would gain performance on, is load times between scenes or something like that. Not really worth the hassle probably. But purely hypothetically speaking, you could install a game in /usr/local/unreal, after the installation move it to /usr/local/unreal_disk. Then create a tmpfs @ /usr/local/unreal, then copy the contents back like this:
cp -Raf /usr/local/unreal_disk/* /usr/local/unreal/
And then run the game.. And then find it hasn't got enough memory left to properly run the game properly ;)

#3. Joseffb on 14 August 2007

Default avatar:JoseffbHow would you use this for gaming? Let's say with a linux client of unreal or along those lines.

#2. Kevin on 19 July 2007

Member avatar: KevinYou're welcome.
The community has given me so much over the last couple of years I figured it was time to start giving back. But if you like it, please social bookmark this article so it can be of help to others too! Thanks!

#1. Nice info on 19 July 2007

Default avatar:Nice infoHi, this is an excelent info Thanks for it!!!