Thursday, 22 April 2010

Fastest way to create ramdisk in Ubuntu/Linux

Fastest way to create ramdisk in Ubuntu/Linux: "

I hope many of you will agree that sometimes it’s really good idea to have some small amount of RAM mounted as a filesystem. It may be necessary when running some bash or perl script that handles, say, thousands of small files so it’s much more effective not to waste computer resources on reading/writing data on hard disk but keep those files directly in memory. This idea is known as Virtual RAM Drive or ramdisk and can be setup in Ubuntu or almost any other Linux distribution using the following commands under root (to become root in Ubuntu use 'sudo -s“):


# mkdir /tmp/ramdisk; chmod 777 /tmp/ramdisk

# mount -t tmpfs -o size=256M tmpfs /tmp/ramdisk/


where 256M is amount of RAM you wish to allocate for ramdisk. It’s clear that this value should be less than amount of free memory (use “free -m“). BTW, if you specify too many MBs for ramdisk Linux will try to allocate it from RAM and then from swap so resulting performance would be very poor.




"

Friday, 19 February 2010

Waiting for a mobile device to die....

The Problem:

I strikes me as my Samsung PMP slowly dies next to me that waiting for a mobile device to die these days is very much like baking.  Like how opening the door on a delicate suflay do check if it's done would ruin it, if I keep waking the Samsung up to check how much is left I simply drain it more.  The screen is the biggest battery drain of all.  So a couple of things wind me up about this situation.  One is that the only connector (apart from the headphone jack) is this proprietary Samsung crap - so I can't plug it into one of my abundant USB ports I have lying around.  For crying out loud, my alarm clock has 2 in it!  The other thing that does my head in, is that these devices that are running out of battery seem to go out of their way to drain themselves telling you.  Thus hastening their demise considerably.

The solution:

Okay, there are two simple answers to end this nonsense.

1. Use standard USB ports everywhere.  If this isn't suitable for the device (due to form factor) then introduce a third form factor to USB (to compliment the two USB already has) that has a slimmer profile.  

Of course a better solution might be to actually adapt the other existing port these devises have.

2.  Many of the devices I'm talking about all have a standard 3.5mm mini-jack... or headphone socket if you will.  So why can't that supply power, just like USB does?   Think of the benefits!  Devices could charge each other!  I wonder what the complications of sending a charge over the same wire that's carrying signal.  If that weren't possible, then it would mean a redesign of the 3.5mm plug/socket spec, which probably makes it infeasible.

Oh well.  My device is still running, and hopefully the suflay is not ruined.  Oh yea, and on these stupid devices that constantly alert about dying... make them smart. Put them in low power mode (at the least dim the screen). Have a semi-awake mode where the first stage wakeup is just a dim power read out.  To wake fully, just press on again.

Bet's on any of this actually happening....


Thursday, 4 February 2010

Temporary tables, union and union all

Temporary tables, union and union all: "Both UNION and UNION ALL use a temporary table to buffer results from all branches of the query. The temporary table buffers all rows before any are returned to the client. For UNION, a unique index is created on the table. For UNION ALL, a unique index is not created. The HEAP (MEMORY) engine is used as long as there are no LOB columns.

The HEAP engine uses fixed length rows. For variable length columns each row allocates the max size per column so a lot of memory can be wasted. When the temp table gets too big, it is converted to a MyISAM table. The switch is done when the size of the table is min(max_heap_table_size, tmp_table_size) (I think that equation is right, wouldn't it be nice if it were simpler?).

In some cases UNION ALL can be used to reduce network round trips when there are multiple SELECT statements that return similar columns: select * from foo UNION ALL select * from bar. While this reduces network latency, it can increase the load on the server from writing the temporary table.

A feature request might be open to not use a temporary table for UNION ALL. Search for more details on this topic.

What happened to the change to not allocate the max length for variable length columns in the HEAP engine? When will it appear in an official release? There is a feature request for it.

And now, a quiz from Domas. What is done for the following?

(select * from foo UNION ALL select * from bar) limit 10

PlanetMySQL Voting:
Vote UP /
Vote DOWN"

Friday, 11 December 2009

Windows 7 Automatic Update Shutdown HELL

So I came in this morning and Windows 7 had rebooted my PC.  I had 7.5GB of applications running across two operating systems (Ubuntu running in a Virtualbox - connected to lots of servers for work).  This made me very mad.  I had updated Windows Update to only download the updates, and let me choose when to install them.  I had installed them - and suffered the annoying "Reboot now for Important updates to be installed"  and I kept postponing it for 3 hours.  The last thing I expected was to come in and find that windows got fed-up asking me and decided to reboot anyway.

Well, I found a way to stop it - and thought I'd share it here as Google seems to return mainly crap about the BETA version rebooting when it's about to expire.  Convenient me thinks - this will mask any REAL issues with Windows 7 rebooting.

Anyway - here's the link:  http://www.howtogeek.com/howto/windows-vista/prevent-windows-update-from-forcibly-rebooting-your-computer/

Wednesday, 2 December 2009

Google Profiles Turn Into OpenIDs

Google Profiles Turn Into OpenIDs: "

Screen shot 2009-11-25 at 12.25.14 PMAs part of its push to go more social, Google has been attempting to unify its various account profiles into one Google Profile. And now it’s more useful. Google’s Brad Fitzpatrick has just tweeted out that Google Profiles can now be used as OpenIDs.


What this means is that you can sign into any site that accepts OpenID simply by using your Google Profile domain. Luckily, a few months ago Google started allowing these profiles to have vanity URLs, like /mgsiegler, instead of the previous /32090329039402903. Chris Messina, a huge proponent of the open web movement, has just sent out a picture of what signing in with OpenID via your Google Profile looks like (below).


Despite its good intentions, OpenID has yet to take off in mainstream usage. The problem, it seems, is largely about presentation. Most people have no idea which of their various accounts can be used as OpenIDs, or really even what OpenID is. Google backing it a bit more with these profiles obviously helps, but will it take OpenID mainstream? Probably not.


More interesting may be the second part of Fitzpatrick’s tweet. “Also, gmail webfinger declares that now too.” It’s not entirely clear what he means by that, but it would seem to suggest that we’re getting closer to being able to use our Gmail addresses as a web ID. WebFinger is a protocol being worked on by Fitzpatrick to allow you to attach information to your email address (in this case, you Gmail address), so it can be used as a solid means of identification.


4134418702_0ffffed62d_o


Update: Kevin Marks (former Googler, now with BT) has pointed me to webfinger.org an example site built by Blaine Cook (formerly of Twitter). The site allows you to easily set up your Gmail account with Webfinger right now. As you can see in the example below from Cook’s account, many of the social networks Cook is a part of are pinned to his email address.


Screen shot 2009-11-25 at 1.02.55 PM



Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.









"

Google Profiles Turn Into OpenIDs

Google Profiles Turn Into OpenIDs: "

Screen shot 2009-11-25 at 12.25.14 PMAs part of its push to go more social, Google has been attempting to unify its various account profiles into one Google Profile. And now it’s more useful. Google’s Brad Fitzpatrick has just tweeted out that Google Profiles can now be used as OpenIDs.


What this means is that you can sign into any site that accepts OpenID simply by using your Google Profile domain. Luckily, a few months ago Google started allowing these profiles to have vanity URLs, like /mgsiegler, instead of the previous /32090329039402903. Chris Messina, a huge proponent of the open web movement, has just sent out a picture of what signing in with OpenID via your Google Profile looks like (below).


Despite its good intentions, OpenID has yet to take off in mainstream usage. The problem, it seems, is largely about presentation. Most people have no idea which of their various accounts can be used as OpenIDs, or really even what OpenID is. Google backing it a bit more with these profiles obviously helps, but will it take OpenID mainstream? Probably not.


More interesting may be the second part of Fitzpatrick’s tweet. “Also, gmail webfinger declares that now too.” It’s not entirely clear what he means by that, but it would seem to suggest that we’re getting closer to being able to use our Gmail addresses as a web ID. WebFinger is a protocol being worked on by Fitzpatrick to allow you to attach information to your email address (in this case, you Gmail address), so it can be used as a solid means of identification.


4134418702_0ffffed62d_o


Update: Kevin Marks (former Googler, now with BT) has pointed me to webfinger.org an example site built by Blaine Cook (formerly of Twitter). The site allows you to easily set up your Gmail account with Webfinger right now. As you can see in the example below from Cook’s account, many of the social networks Cook is a part of are pinned to his email address.


Screen shot 2009-11-25 at 1.02.55 PM



Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.









"

Google Profiles Turn Into OpenIDs

Google Profiles Turn Into OpenIDs: "

Screen shot 2009-11-25 at 12.25.14 PMAs part of its push to go more social, Google has been attempting to unify its various account profiles into one Google Profile. And now it’s more useful. Google’s Brad Fitzpatrick has just tweeted out that Google Profiles can now be used as OpenIDs.


What this means is that you can sign into any site that accepts OpenID simply by using your Google Profile domain. Luckily, a few months ago Google started allowing these profiles to have vanity URLs, like /mgsiegler, instead of the previous /32090329039402903. Chris Messina, a huge proponent of the open web movement, has just sent out a picture of what signing in with OpenID via your Google Profile looks like (below).


Despite its good intentions, OpenID has yet to take off in mainstream usage. The problem, it seems, is largely about presentation. Most people have no idea which of their various accounts can be used as OpenIDs, or really even what OpenID is. Google backing it a bit more with these profiles obviously helps, but will it take OpenID mainstream? Probably not.


More interesting may be the second part of Fitzpatrick’s tweet. “Also, gmail webfinger declares that now too.” It’s not entirely clear what he means by that, but it would seem to suggest that we’re getting closer to being able to use our Gmail addresses as a web ID. WebFinger is a protocol being worked on by Fitzpatrick to allow you to attach information to your email address (in this case, you Gmail address), so it can be used as a solid means of identification.


4134418702_0ffffed62d_o


Update: Kevin Marks (former Googler, now with BT) has pointed me to webfinger.org an example site built by Blaine Cook (formerly of Twitter). The site allows you to easily set up your Gmail account with Webfinger right now. As you can see in the example below from Cook’s account, many of the social networks Cook is a part of are pinned to his email address.


Screen shot 2009-11-25 at 1.02.55 PM



Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.









"

Saturday, 28 November 2009

Restore a Single Table From mysqldump

Restore a Single Table From mysqldump: "Ok, this is inspired by another post on the same subject. I would have just commented, but there was a required signup. Ew.


While the method there with juggling grants works fine, I'd like to highlight another way that I think is much simpler: use grep & sed to create a file which contains only the table you want to restore.


mysql> show tables;
+------------------+
| Tables_in_gtowey |
+------------------+
| t1 |
| t2 |
| t3 |
+------------------+
3 rows in set (0.00 sec)


# mysqldump gtowey > dump.sql

Given 3 tables in a mysqldump file, lets restore just t2


# grep -n 'Table structure' dump.sql
19:-- Table structure for table `t1`
40:-- Table structure for table `t2`
61:-- Table structure for table `t3`


Now just use the line numbers to extract the table with sed:

sed -n '40,61 p' dump.sql > t2.sql

That's it, you have everything you need with no cleanup of mysql grants to do afterward.

PlanetMySQL Voting:
Vote UP /
Vote DOWN"

Monday, 23 November 2009

Fast restore one database from a dump of ALL databases

Fast restore one database from a dump of ALL databases: "There have been times that I needed to restore one database from a dump file that contains all of the databases from the server. There are a few ways to accomplish this goal. One way would be to grep through the entire file for the table schema and insert statements. There are some problems with this method in some environments. For example, what if a table name in the targeted databases had the same name in another database on the same instance? The logic for grep now gets increasingly more complex.



It’s a good thing that the developers at MySQL already thought of this and implemented it, --one-database. In the example below I have created 4 databases, test, test[1-3] and filled each of them with 10K rows. I run mysqldump for all databases and put them in a file called test.all.sql then truncate test2.tbl1. Using mysql with the --one-database option I restore the data to its original state.



Example:



mysql> select min(anum), max(anum) from test.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 0 | 10000 |

+-----------+-----------+

1 row in set (0.01 sec)



mysql> select min(anum), max(anum) from test1.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 10001 | 20000 |

+-----------+-----------+

1 row in set (0.01 sec)



mysql> select min(anum), max(anum) from test2.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 20001 | 30000 |

+-----------+-----------+

1 row in set (0.01 sec)



mysql> select min(anum), max(anum) from test3.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 30001 | 40000 |

+-----------+-----------+

1 row in set (0.01 sec)



shell> mysqldump -uroot -S mysql.sock --opt --all-databases > test.all.sql



mysql> use test2;



mysql> select count(*) from tbl1;

+----------+

| count(*) |

+----------+

| 10000 |

+----------+

1 row in set (0.01 sec)



mysql> truncate table tbl1;

Query OK, 0 rows affected (0.05 sec)



mysql> select count(*) from tbl1;

+----------+

| count(*) |

+----------+

| 0 |

+----------+

1 row in set (0.00 sec)



shell> mysql -uroot -S mysql.sock --one-database test2 < test.all.sql



mysql> select min(anum), max(anum) from test.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 0 | 10000 |

+-----------+-----------+

1 row in set (0.01 sec)



mysql> select min(anum), max(anum) from test1.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 10001 | 20000 |

+-----------+-----------+

1 row in set (0.01 sec)



mysql> select min(anum), max(anum) from test2.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 20001 | 30000 |

+-----------+-----------+

1 row in set (0.01 sec)



mysql> select min(anum), max(anum) from test3.tbl1;

+-----------+-----------+

| min(anum) | max(anum) |

+-----------+-----------+

| 30001 | 40000 |

+-----------+-----------+

1 row in set (0.01 sec)



As you can see test2.tbl1 has the original data back in and we did not have to write our own script to accomplish this.
PlanetMySQL Voting:
Vote UP /
Vote DOWN"

Tuesday, 17 November 2009

mk-query-digest now understands HTTP

mk-query-digest now understands HTTP: "

You used to use mk-query-digest to aggregate and report on MySQL’s slow query log. Then it got the ability to grab query events from polling SHOW PROCESSLIST. Next we thought, really, how hard can it be to implement the libmysql wire protocol, so we can sniff TCP packets? … it’s hard, but not that hard as it turns out. But why stop there, why not implement memcached protocol too? I think you can see where this is headed.



So now mk-query-digest is a tool that can understand and “do stuff with” a variety of query/response types of information. The latest is HTTP. HTTP traffic is just a query-response flow of events, perfectly suitable for response-time analysis. Example:



baron@kanga:~$ mk-query-digest sample-http.txt --type http
# 1.6s user time, 100ms system time, 14.20M rss, 17.07M vsz
# Overall: 56 total, 30 unique, 1.27 QPS, 0.06x concurrency ______________
# total min max avg 95% stddev median
# Exec time 3s 746us 563ms 49ms 293ms 110ms 3ms
# Time range 2009-11-14 07:04:50.997506 to 2009-11-14 07:05:34.969818
# bytes 0 0 0 0 0 0 0
# Transmit 10s 0 1s 176ms 777ms 301ms 28us

# Query 1: 0.28 QPS, 0.02x concurrency, ID 0x198704D47EE1BF0C at byte 103987
# pct total min max avg 95% stddev median
# Count 21 12
# Exec time 24 670ms 3ms 563ms 56ms 13ms 150ms 12ms
# Hosts 12 12.38.8.23... (1), 121.44.246... (1)... 10 more
# Time range 2009-11-14 07:04:51.763526 to 2009-11-14 07:05:34.969818
# bytes 0 0 0 0 0 0 0 0
# Status co 1 200
# Transmit 62 6s 0 1s 514ms 900ms 376ms 580ms
# Virtual h 1 www.mysqlperformanceblog.com
# Query_time distribution
# 1us
# 10us
# 100us
# 1ms ####################################
# 10ms ################################################################
# 100ms #########
# 1s
# 10s+
get www.mysqlperformanceblog.com/feed/\G

... snip a bunch of output ...

# Rank Query ID Response time Calls R/Call Item
# ==== ================== ================ ===== ======== ================
# 1 0x198704D47EE1BF0C 0.6700 25.5% 12 0.0558 GET www.mysqlperformanceblog.com/feed/
# 2 0x2CBDA396697F8145 0.3408 13.0% 1 0.3408 GET www.mysqlperformanceblog.com/
# 3 0x2E18EB8C0CD9AED9 0.3100 11.8% 1 0.3100 GET www.mysqlperformanceblog.com/page/2/
# 4 0x4686B61E6556B753 0.3042 11.6% 1 0.3042 GET www.mysqlperformanceblog.com/
# 5 0x247CABBCB1B76C01 0.2705 10.3% 1 0.2705 GET www.mysqlperformanceblog.com/page/28/
# 6 0x8C239A43A9C80FD2 0.2373 9.0% 1 0.2373 GET www.mysqlperformanceblog.com/
# 7 0x4D4095C546E65CD4 0.1959 7.5% 1 0.1959 GET www.mysqlperformanceblog.com/2008/11/26/
# 8 0x49CC22FAC68CD475 0.1906 7.3% 1 0.1906 GET /favicon.ico


This is suitable for lots of things. We’re trying to look at the most useful protocols, because the variety of inputs is really unlimited; we could implement almost anything that fits into the notion of query and response. For example, the memcached protocol is becoming something of a lingua franca for a lot of different systems, so there’s a big value-add. HTTP has been used a long time as a transport layer for REST, SOAP, and so on (CouchDB anyone?). Valid, and interesting, suggestions are Sphinx, PostgreSQL, and Gearman. (Please offer to sponsor any that you want to see.)



Back to HTTP: implementing it gives an easy way to measure website response time, including useful things like 95th percentile goodness. And from there, you can drill down into the performance of the work done for these requests. If you want to get really fancy, you can even capture some samples of netstat at the same time as you tcpdump traffic for HTTP, memcached, and MySQL — so you can blame database queries and memcached requests on specific HTTP requests!



Related posts:

  1. An alternative to the MySQL Query Analyzer MySQL just
  2. Seeking input on a badness score for query execution Suppose th
  3. Analyze and optimize memcached usage with Maatkit Ryan poste


Related posts brought to you by Yet Another Related Posts Plugin.


PlanetMySQL Voting:
Vote UP /
Vote DOWN"

Monday, 16 November 2009

QuickTip: Make life easier with ssh-copy-id

QuickTip: Make life easier with ssh-copy-id: "

How many times have you ran through this series of events?



$ cat ~/.ssh/id_dsa.pub

...copy output to clipboard...

$ ssh myhost

...enter password...

myhost$ vi ~./ssh/authorized_keys

...paste public key and save...

myhost$ exit


Thanks to bash's tab completion, I happened upon ssh-copy-id. Instead of all that above, just do this:



$ ssh-copy-id myhost

...enter password...



You're done!


read more



"

Thursday, 12 November 2009

How to tweak your Linux machine’s history command

How to tweak your Linux machine’s history command: "Linux

If you are somewhat familiar with using the Linux or UNIX command line interface it is quite likely that you have used the history command. For those not familiar with it the history command allows you to search through the commands you have already run on your computer’s command line.


An example of its usage is:


# history | grep “kill”


This will search for a command containing the term kill through the history of commands you have run on your computer. This is a basic example of the history command. There are certain limitations in this command. Let’s take a look at how you can tweak it a bit to work more effectively for you. One of the limitations I find bugging me quite often is that if you are running two terminal session at the same time the commands from the first session are erased from the history of commands. Not the smartest way the command should work, I think. Well let’s fix it then.


Log in to your account and launch a shell terminal. In your user’s home directory you should find a file called

.bashrc. If this file does not exist, create it. To open the file use a command like the following, replacing user with your username:

# vim /home/user/.bashrc


Add the following lines to the .bashrc file without any changes whatsoever:


shopt -s histappend

PROMPT_COMMAND=’history -a’


And voila, problem solved. Henceforth all your commands from single sessions as well as simultaneous sessions will be written to the command history file, and no overwriting will happen.


Another useful change I like to make to the history command’s configuration is removing duplicates from the history of commands. This way I get access to a lot more data and get it quicker. Open the .bashrc file once again and add the following lines in a new line:


export HISTCONTROL=”ignoredups”

export HISTIGNORE=”&:ls:[bf]g:exit”


Save the file and exit the text editor. Now the history command will check to see if a command already exists in the log file or not before adding any new information. A last modification I make to the history command’s configuration is adding spell check. Add the line shown below to the file .bashrc to help the history command check for spelling errors in command and suggest a valid command:


shopt -s cdspell


Save and exit. Now your Linux machine will be able to figure out when you type gerp instead of grep.


---
Related Articles at Simple Help:







"

Sike's shared items