Friday, September 23, 2011

Modify encrypted swap partition in Ubuntu

If you want to resize, merge, split, move your partitions, you may have to modify your encrypted partition too.

If you haven't got it, install GParted to view and manage your partitions.

Before you do anything serious, make a rescue disk in a USB thumb flash drive, and make sure that your computer can boot from it, and you know how to.  In Ubuntu a rescue/recover disk is actually a full sized Ubuntu operating system on trial.  You can use it straight from the flash drive, or install Ubuntu into your hard drive for a dual boot or pure Ubuntu system.  Just don't do that and keep using it directly from flash drive.

The latest Ubuntu have GParted installed, and make sure that your thumb drive have it.

I hate to say that you should back up you data before you modify your partitions.  But that's not very practical for most, unless you have a new bigger hard drive, or a huge external drive.  GParted is pretty safe and I lived to tell you the story.

If you didn't setup your swap, I didn't, you may find where it is in the file /etc/fstab.  You will find a line with the word swap.  For a swap partition it will be at /dev/sda2 etc, but identified with the partition's UUID, a global unique code.  For a swap file, it will be typically be at /swapfile.

The size of the swap file or partition will be about one to two times your system RAM.  If you have ample RAM and disk space, the recommended size is equal to your RAM.  But the old school of twice the RAM size doesn't matter.  Now disks are half to one Terrible lot of Bytes.

Especially if you encrypted your swap file, you cannot be sure that it's functional because in GPart it show up as an unknown partition and unmounted.  Don't let it fool you.

First you turn off swap:

#swapoff /dev/sda2

for swap partition or if you have a swap file

#swapoff /swapfile

Edit the /etc/fstab file:

#sudo gedit /etc/fstab

and comment out the swap file entry.  If your swap isn't encrypted, now it would be a normal partition (if you reboot) and you can do anything about it with GPart for example.  You can delete it or your swap file.

If your swap is encrypted, you will not be able to boot again.  You have to remove the mapper.  You can see at /dev/mapper what the mapped partition is, eg, cryptswap1. To remove it:

#dmsetup remove cryptswap1

Without it, you OS cannot boot even in recovery mode, as the mapper is looking for the swap partition that do not exist or defunctional one way or the other.

To setup the swap partition again.  Create a partition in GPart if you don't have one already.  Keep it unmounted.  Format it to linux-swap.

In /etc/fstab, uncomment the old swap partition line.  Instead of the UUID, you can just use the device name, /dev/sda2 for example.

You may be able to do

#swapon /dev/sda2

But if you reboot you will sure have the swap partition active.

Now you can use the ecryptfs utility to encrypt the swap file again.  You can see your swap at /dev/mapper and the corresponding line added at /etc/fstab.

Remember to comment out the line for the previous swap partition in /etc/fstab.  The ecryptfs utility does not do it for your.

You can use the system monitor to see how much and how often your swap partition is used.

Wednesday, August 24, 2011

Ubuntu startup disk on USB flash drive refuse to boot

I have created a startup disk on USB thumb drive for an ancient desktop.  I never tried to boot it up.  Nowadays, basically it's a full copy of virgin OS on flash drive.  For this reason you shouldn't bother with Windows recovery.  Have Ubuntu dual boot and you can read and write your drives no matter what.  A flash drive is extra protection when both boot fails.  A 2 GB drive is too cheap for the postage.  If you have Frys or Micro Centers, you can pick it up dirt cheap.  

For strange reasons the flash drive wouldn't boot.  I searched the Web.  All the problems and solutions are not relevant to me.  There's only one person saying that must be memory.  Of course, but memory errors are the most spectacular.  I'm not having anything spectacular.  As a last resort I tested my memory using the memtest that comes with Ubuntu.

It wasn't surprising that I have at least a memory error.  I think I spotted errors long ago.  I don't believe it because it passed the BIOS tests, and Windows and Ubuntu boots well. 

For no good reason I swapped the memory banks.  It was scary as the motherboard refuse to boot again.  It boots if I swap them back.  So the simple memory tests test only some of the bytes, not all the bytes.

So I went to Micro Center to pick up some cheap memory.  Memtest shows no error and everything boots, including the USB flash drive.  Amen.

The life time guarantee on the memory is basically useless.  By the time good memory fail, it will be obsolete, if not, dirt cheap.  If you are still using your obsolete system, for memory you cannot wait for the return authorization and wait for them to sent you the memory.  And if you bother to pull out the memory, it's worth while to upgrade to bigger memories.

There comes a time when obsolete parts cost more than new parts, for the hard to find factor.  Basically this is driving you to give up your obsolete system and upgrade.

Epson scanner warmup problem in Linux

Today I reached the holy grail, the ancient motherboard and ancient Epson scanner is working under Ubuntu, the last device that refuse to work.

With the kernel upgrade, the know problem of my most ancient motherboard is gone all of a sudden.  The motherboard temperatures can be detected and I can use it to run something hot, rather than sitting pretty.

My ancient Epson USB scanner always work under Windows.  But my other things wouldn't work nicely in the ancient mother boards.

Firstly, in Ubuntu, you need the sane package installed for scanners.  Try type the command xsane.  If you don't have it installed, follow the instructions to install it or just fire up the package manager.  The sane package will be chain installed in order to get the GUI front-end working.

Type the command

#scanimage -L

to check if your scanner is recognized.   For some Epson scanners, you can see the name of the so called backend, some sort of driver - plustek.

My scanner is recognized, responded, but hanged after scan started with no messages.

Ignore what Epson have to say and their recommendation of drivers for Linux.  It is way too out dated.

Go to /etc/sane.d

Edit plustek.conf file (sudo)

Change
option warmup -1
to
option warmup 60

Then my scanner works.  If they still don't, edit all the epson*.conf files.  Comment out the scsi lines.

Fire up your scanner with xsane and be patient.

Tuesday, August 23, 2011

What is WiiXplorer?

You don't really have a clue after reading the long list of features.  But it is a must have.

It can be installed directly via the Homebrew Browser (an app on Homebrew channel).  So you don't need to do anything like touching the SD card.  I installed it after I installed USB Loader.  It works without a hitch.  I suppose it will work without the USB Loader.  I have everything compatible to Wii 4.3.

Now that you need the USB drive to be at the Wii to backup all the games, you don't want to move the drive upstairs and downstairs to the PC and the Wii.  Two separate drives aren't necessary better even if you have the money.  You need something to connect your PC to the USB drive at the Wii.

The Wii is a pretty good SDHC card reader (and writer).  You can watch your pictures and videos right on the big screen after a trip.  If you can copy the card to your PC, you don't need a separate card reader.

The WiiXplorer is basically a GUI file manager, like that of any other OS.  It's behavior like Windows Explorers and nautilus(?) in Ubuntu.  Basically that's the tasks you do daily the most, copy, move, delete, rename files and folders.

The multi device means the SD card drive, USB drive, and SMB share on the PC over WiFi is supported.  It should also support USB flash drive, but I didn't tried.

There is an icon at the top left hand corner to show what device is being displayed.  If you click on it, there's choice of SD, USB, and SMB etc.

The USB drive connected to the Wii is NTFS formatted (Windows).  My desktop is of course ext (Linux), but wrapped up with SMB.

It's the same SMB setup as mplayer-ce.  But you can do it via the GUI when you run WiiXplorer.  Remember to uncheck readonly at the PC end if you want to be able to write back to the PC drives.


So you can copy/move files from any device to any device.  And you can sit at the Wii and do everything with the Wiimote, without going to your PC, as long as it is on for the SMB server.

If you download something to the PC into the SMB share folder, you can move it to any storage on the Wii.  You can also move anything on the Wii storages into your PC.  With this thing, you never need to go up and down stairs to install something.

But moving files over WiFi is pretty slow if you have 4GB of data.  The speed is about 500 MB/s.  You need 2000 sec to move 1GB of data.  It's not too bad.  If you want it immediately, you can carry the USB drive around, the speed is at least 20MB/s at the lower end.  Otherwise you can start moving something when you are not using the Wii for an hour or two.

If you setup the Wii as an ftp server, you can sit at the computer and do everything too.

WiiXplorer also launch installed Homebrew applications according the file type.  But it didn't work for me.  If the image display works, then it's a compliment to mplayer-ce.

Usng WiiXplorer is obvious, except that the right click on a file is achieved via the + button.  You use this button to bring up the copy-paste menu.

Wii media center limitations

Last time we talked about streaming movies to the Wii directly from your computer, via wifi connection.

The Wii is not HD yet, and the connections are standard RCA jacks.  You can have adapters for your PC or HDTV for cheap, but the resolution is limited to about DVD resolution of 480p, that of the Wii output.

The mplayer-ce can play almost any formats in any resolutions, but the Wii isn't fast enough for playing HD stuff.  Down conversion from HD to SD is pretty intensive processing, decode HD stuff and then recode.  You have to do it in your PC first.  The processing takes roughly the same order of time as the movie play time, depending on your CPU.

Setting up Wii as a mega game center

Instead of swapping DVD games, you can put them all on a USB drive.  That's mega.  You can also "exchange games" via the Internet this way.

The problem for the Wii is the versions.  Software modifications are very specific to the firmware version of Wii.  Since these softmods aren't updated that often, you may be using out of date instructions.  There's still a possibility of bricking the wii.

The other problems are trolling.  You never know what the purposes of some instructions, and the mentality of some writers.

Let me tell you a story.  I overheard a kid said to his father, "You said we will go to the game shop again to get a new game.  That was last January."  So the kid probably got the Wii for Christmas or Thanks Giving, at the sales or not.  Then he got a game probably the next year at New Year sales.  Then no more games for another year, probably.  So, for whatever reasons, please minimize the trolling.

So if your Wii is virgin, update to 4.3U/E/J.   U for US, E for Europe, J for Japan etc.  You can do everything in 4.3.  If you already have homebrew something in your Wii, I don't think updating will brick it.  But you will have to install homebres all over again.  For this reason, you want to disable automatic updating once you complete installing everyting homebrew for 4.3.

First the trusted instructions are here at GAF, pretty good, very detailed but not too specific.  The order of installation is:

Wii 4.3
Homebrew
IOS236
d2x
Configurable USB Loader

Homebrew is simply a simple way to run other applications and installers.  Almost all softmods are IOS, modifying the I/O system of the Wii, say, instead of reading the DVD drive, read from the USB instead.  d2x handles with some ways that games uses I/O.  The USB Loader manages the games and play them.

Firstly, about the security.  These installers load a lot of things directly from websites and Nintendo.  So you have to trust them.  If you have internet filters for your kids, typically these ware sites and even game sites are blocked by default.  Then you MAY have to turn your firewall off to your Wii, and only your Wii.  You can do it easily on your Wifi router by setting the DMZ option on your Wii - demilitarize zone, meaning no protection.

After installing everything, you should set the firewall to normal.  The homebrew software will still download from the web, but this is normally allowed.  There is no guarantee what is in your softmods.  For example, the easiest method to install Homebrew is to use the letterbomb method, without actually buying an old game.  But your IP and the MAC of your Wii is given away.

If you If the installation complains that it cannot download a file, search for the file directly on the internet, download it, and put it in the right place on the SD card as directed.  This probably won't happen if your Wii is unprotected from the Internet.  The installers actually check hash signatures, so you probably won't download the wrong files into your wii.

Before you do anything, it's best to go to Homebrew channel, get to BootMii by the Home button.  Then save 512 mb of NAND memory onto the SD card.  In BootMii, the wiimote won't work, use the Wii power button as next, and the reset button as select.  If anything goes wrong, you can load the NAND back to the Wii and all your installations are nulled.

After installing IOS236, you should get the hell out without any further steps, step 2.  To get out of step 2, you push the 2 button, confusing, isn't it?  Otherwise you need to insert a real game, start it, and install IOS236 again.

Something is missing in the USB Loader instructions.  I don't know what the hell he is talking about.  Just download the v69 version directly.  It works as instructed.

I advise using NTFS for file format, which is the usual format that comes with USB drives, which is used in Windows, Linux compatible.  You have to add two lines to the config.txt file.  Then you don't need to add another FAT32 petition.  Just remember that in NTFS, filenames are case sensitive, while the old FAT are not.

The important thing is that, USB Loader (and a few others with similar names) put everything on the folder wbfs at root of your USB drive (single or multiple partitions).  Otherwise the loader won't see them.

You can only plug the USB drive into USB1 socket, the one near the bottom or near the edge.

The loader will backup your DVD game to the USB drive via the + button.  It takes a while for all 4 Gb of data.  The filename will be GAMEID.iso inside the wbfs folder (or a few other alternatives).

If you somehow get DVD image files .iso files, you must lookup the correct GAMEID from the internet such as http://wiitdb.com/.  Rename your file to GAMEID.iso and put it right under the wbfs folder.  Or else the loader won't see the game.

If you play the same games often, you can use the SDHC cards too.  You need an 8 GB for one game, and a 16 GB card for 3 games.  USB flash drive should work.  You don't even need a card reader for your PC.  But with the USB socket at the back of the Wii, I find it not that convenient.

I have a very old SD card (up to 2 GB), but I never come across the need to use it in installing all the homebrew software.  All SD cards are now SDHC.  Anything other than the biggest capacity you can get it on the net for cheap.  So cheap that the postage is significant.  You can pick these things up like trash in shops such as Fry's and Micro Center, if you are lucky to live near one.

The best format for external HDD is NTFS.  HDD is also the cheapest, but some adaptors of old internal hard drives don't work.  As for the games, you simply download the .iso DVD image file and put it under the wbfs folder.  Some are compressed into .rar for example after "scrubbing".  You extract back the .iso file after download. 

The USB loader can also read .wbfs files.  But the HDD should also be formated as wbfs.  If your HDD format is NTFS and you download a .wbfs file into the wbfs folder, it will play, but the USB loader will ruin your whole HDD.  You don't discover it until the whole disk of data is corrupted.  I have unrecoverable errors probably due to a combination of back sectors.  Even after complete reformatting, the HDD isn't "clean" as reported.  So I reserve 100 GB for games in one partition.  The others disk space I can safely create a clean partition and use it for backups.

Sync your folder in the cloud with your local drive

For backups, you sync your local folder to some servers in the cloud.  Since only the updated files are copied, you can run the command via cron automatically, daily or hourly.  You can do the same to encrypted files if you use ecryptfs.

For bittorrent downloads into the cloud, you can automatically sync the downloads to your local drive.  With automatic sync, it's as good as downloading directly, without the hazzle of starting bt client on your computer.  But the bandwidth usage increases by 50%, because the files got transferred out of the cloud twice, one for seeding and one to your computer.

The easiest way is using Ubuntu in EC2 and locally, which I am.  First to copy a file or whole folder, from EC2 to hard drive.  Assume you have a folder called downloads in EC2, and your default username there is ubuntu.

scp -i /home/yourusername/yourkeyfile.pem ubuntu@ec2-70-19-168-131.compute-1.amazonaws.com:downloads/foldername /home/yourusername/downloads

The advantage is that there is a counter for the percentage completion.  You can custom the filenames and folder names.

If you sync the files, a single command is all you ever need, and you can put it into cron and forget about it.

rsync -e "ssh -i /home/yourusername/yourkey.pem" -av --exclude '*.part' ubuntu@ec2-33-55-444-222.compute-1.amazonaws.com:downloads ~

For bittorrents, you exclude the partial downloads and never waste bandwidth.  The disadvantage is that the command is completely slient.  You have no way of knowing the progress unless using other means such the system network monitor.  Also the partial file is hidden somewhere, so you cannot take a peep first.