Install a custom Android OS

In this post I am going to try to explain some concepts that I think are not really well explained out there regarding installing a custom Android OS version. And showing along the commands to do it from scratch.

It’s important to say that I am not an Android expert at all, I am just posting what I learned in the way while installing a custom Android version on my phone, so this is going to be just a very high level overview. If you are going to do this on your phone, please first do a thorough research on how to do it specifically for your phone.

Rooting

Rooting is the process of getting admin rights on your Android OS, the same as being admin in Linux or Windows. Manufacturers usually don’t give this type of access so you don’t mess your phone’s software.

Is important to note that this has NOTHING to do with installing a new Android version on your phone. To have root access is just being able to do admin stuff on your current Android version.

Unlocked bootloader

The bootloader on your phone is like the basic program that is in charge of booting the Android operating system. It would be the same as the BIOS on a PC computer. It is the first program that is run when the phone is switched on, searches for an operating system and boots it.

Again, manufacturers usually lock the bootloader of your phone, so that you don’t try to do fancy things on your phone.

So to be able to install a new Android version, the first thing you need to do is unlock the bootloader. Some manufacturers provide a way to do it, usually you register on their web page and they send you a code and the instructions to do it. Not without previously warning you that you will loss the warranty. Just google “unlock bootloader [phone brand and model]” for specific instructions for your phone.

Some other manufacturers directly don’t allow to unlock it, in this case there is usually developers around the world that have already posted a patch to be able to do it anyways .

Phone modes, fastboot and first steps

To be able to install a new Android OS to your phone, you will need to connect your phone to your PC and use a couple of command line programs called fastboot and adb. These programs come with the Android SDK, the individual Android SDK package is called “platform-tools”. There is no need to download the whole android SDK which is about 2 GB, you can just download this package, which is around 10 MB. It can be found here: https://developer.android.com/studio/releases/platform-tools.html, yes it is on the releases part of the android site! Certainly it looks that these people don’t want to make things the easy way.

So you need to do 2 things before attempting to install a new Android OS:

First: unlock your bootloader (explained above)

Second: boot your phone in bootloader mode (some people call it “fastboot mode”, because is the mode that you need to be in to be able to use the fastboot program) to be able to use the fastboot program with your phone. There are certain special button combinations that you need to do depending on your phone, usually a combination of power and volume up/down, sometimes also connecting the USB cable of your computer while pressing a certain button. Google the specific one for your phone.

Or you could also just plug your phone (enable usb debugging from your Android settings) and do:

>adb reboot bootloader

This should always work.

You can tests that the phone is ready to accept fastboot commands with the command:

>fastboot devices

and a description of your phone model will appear.

Then you can execute:

>fastboot -h

and a help page is displayed, with all the commands and options that are availabe for fastboot. Besides “fastboot oem unlock” and “fastboot oem lock” which are the commands (with some variations) that you need to use to unlock and lock your phone. The real star here is this option:

flash [ ]

So the “flash” word appear here, flashing seems to be a very sophisticated word but is more or less the same as copy, meaning that you will copy a filename on your computer to a partition in your phone.

Android partitions

Before proceeding, let’s go to what are your Android phone partitions, which are very similar to standard Linux partitions:

/boot – is the partition where the kernel and the ramdisk resides, this is the partition needed to boot Android. You don’t need to know what a kernel and a ramdisk are, this is just for information purposes.

/system – is the partition where the android system is actually installed.

/recovery – is the partition where the recovery program resides. If you boot into “recovery mode” the bootloader gives control to this partition. More on this later.

/data – is the partition where everything related to the user is stored, like installed apps, messages, etc.

/cache – is the partition used for the caching of frequently accessed data and app components.

Flashing

So basically what the flash command of the fastboot program does, is copying some file on your computer to one of this phone partitions. This file is usually an img file, which is similar to an iso file, meaning that it is a file intended to be copied bit by bit to a media device, like the iso file is intended to be copied bit by bit into a CD. This way the partition will become an exact image of the img file. Is not that the img file is copied to a partition with a filesystem, is that the entire partition will become a copy of the img file. This kind of copying in the android world is called flashing. In the Linux world to do this kind of copying one uses the “dd” command.

Now that we know what flashing is, we need to know what to flash.

The ROM files

The Android OS is developed by Google and it is open source. As phones are not yet very standarized and the Android project is not mature enough yet, this core version of the Android OS is not ready to install on every phone, like Windows or Linux distros are ready to install (more or less) on every PC computer. There is some work needed to be done on top of the OS to make it compatible to a specific device. CynagogenMod was an open source project intended to do this, customize the base Android OS for a vast array of phone models, but they discontinued the project last year. Luckily another project called Lineage OS, got the source code of CynagogenMod, and now they are doing this work. Their web page is: lineageos.org

Go to downloads and see if they are supporting an OS image for your phone, if not, you will have to google where you can get one, a great place for support is: www.xda-developers.com

These customized Android OS versions are called ROMS.

Once you find the correct ROM file containing the customized Android OS version for your phone(it is usually a zip file), then it is recommended that you also download the Google Apps package (another zip file), this file contains Google Play and other commonly used Google applications. It can be downloaded from the open gapps site: opengapps.org. Check your ROM developer instructions to know which version you should download.

So you will first flash your ROM zip file and then apply on top the gapps zip file.

If you have an external card, it is advisable that you copy these 2 files to it, before going into the next step.

Custom recovery tool

If you have an external card, the standard way of installing a new Android OS (in the Android world is called flashing a ROM), is first to download a custom recovery tool that allows you to do lots of fancy stuff, including flashing ROMS.

There are 2 main custom recovery tools: ClockworkMod (CWM) and Team Win Recovery Project (TWRP). TWRP is touch based and has more features, so if it is available for your phone is recommended.

You can google them and find custom image files to install them on your device.

To install (also called flash, because the image is being copied to a partition on your phone) any of these custom recovery tools on your device, you have to use fastboot. From your PC launch a terminal, go to where your fastboot is installed and:

>fastboot flash recovery either_twrp_or_cwm_file.img

Writing in the recovery partition of your phone, the image file for the specific recovery tool that you have just downloaded.

Then you have to reboot into custom recovery mode, which means rebooting from the /recovery partition from your phone that you have just flash. You can do so by pressing a certain button combination or by going to bootloader mode and then to recovery mode, or by using the adb tool.

Once in the custom recovery tool, you can flash the Android OS ROM and gapps zip files. 

Direct install (not needed if using custom recovery tool)

If you don’t have an external card or don't want to install the custom recovery tool, then you have to install the ROM directly with fastboot from your computer by first erasing the userdata and cache partitions with:

>fastboot -w

flashing the ROM file:

>fastboot update downlodade_rom_file.zip

and installing google apps:

>fastboot update gaaps_file.zip

(Note: I haven’t tested these last 2 commands, I installed the new Android version via the custom recovery tool)

Summing up

1. Download the ROM zip file and google apps (gapps) zip file from your computer to your harddrive

2. Connect your phone to your PC and copy these 2 files to the sd card.

3. Reboot the phone in bootloader mode, and use fastboot to unlock your bootloader

4. Install a custom recovery tool also with fastboot. These can be: TWRP or CWM.

5. Reboot your phone into the custom recovery tool and flash the Android OS ROM and gapps zip files.

After doing this and adjusting some steps based on what the ROM publisher recommends, you will have a brand new Android version on your phone. The difference at least in my case was HUGE, is much more faster, and it works fantastic, much better than high end new phones, even though my phone is at least 4 years old, and it was not one of the more expensive ones back then.

UEFI system boot up and GPT partitioning system explained - with an example using parted

I am writing this post in an effort to explain the new UEFI booting system, because for some reason I have found very difficult to have all this simple concepts explained in a concise way out there.
So I will attempt to explain them here in a very high level and simple way.

I will be using the parted program as a way to explicitly show how the partitions are created in an empty hard drive. Parted only runs on Linux, but it can be used from a Linux bootable cd/dvd/usb. Nowadays most Linux distributions have a usable system that runs with the installation cd/dvd/usb.
This part is more directed to Linux users who are usually more used to the command line, but the concepts are operating system independent.

I prefer to use parted in the command line to be able to leave the concepts clearer by following the steps one by one, and not use gparted or some other graphical frontend that does a lot of "magic" by itself.

To run the parted program, go into the command line and type:
> sudo parted /dev/sdX 
(being X the hard drive letter, usually 'a')

You can see the parted reference with the 'help' command:
parted> help 

And the parted reference for a specific command with:
parted> help [command]

Regarding hard drive partition organization and booting, there are 3 main new concepts, all named after their initials:

GPT (Guid Partition Table)

GPT is the new partition table system. Before it, "msdos" was usually used on desktop computers, and all the information regarding partitions was stored in the MBR (Master Boot Record).

Using the parted program, in an empty unpartitioned hard drive, the command "mklabel" creates this new partition table system in the hard drive like this:
parted> mklabel gpt
Please take into account that if the hard drive is not empty this will erase all the information in it!!


ESP (Efi System Partition)

ESP is a special partition formatted with a fat32 filesystem with the "esp" flag added to it that usually goes at the beginning of the disk and has a size of around 100~512 MB.
This partition will have (after operating system installation) the bootloaders for the different operating systems.
With the old system, the bootloader program needed to be in a specific small part of the MBR with very limited space, making it hard for bootloader programmers to be able to fit the program in this small space.

This partition can be created with parted like this:
parted> mkpart primary fat32 1049KB 525MB
This basically tells parted to create a fat32 partition that goes from the 1049KB to 525MB (around 524 MB size) part of the disk.
I used this arbitrary numbers because they were the numbers that were on my laptop disk, but any numbers can be used as long as the partition has a minimun of 100 MB.

This command only creates the partition but does not create the actual filesystem, to do that under Linux, one can use the "mkfs.vfat" command like this:
> sudo mkfs.vfat -F32 /dev/sdX1
(note that you have to quit parted first)

Then the disk can continue to be partitioned...


UEFI (Unified Extensible Firmware Interface)

UEFI is the new standard for booting which uses the ESP partition described (and created) above. It came to replace the old MBR booting.
The UEFI system is available in newer computers (usually it can be enabled or disabled in the system setup, if disabled it goes back to MBR booting), when the computer boots, the UEFI system (built into the motherboard) looks for an ESP partition, search for the bootloaders there, and transfer the control to them. The bootloader is then in charge of booting the correspondent operating system.

Once the operating system is installed, one can browse this partition with a file explorer, and see that the installer automatically put the bootloaders there. For example after the installation of Ubuntu Linux, the grub bootloader is on: "/EFI/ubuntu/grubx64.efi"

Usually on UEFI systems, the bootloader to use can be selected from the system setup, this makes possible (although not very convenient) in the case of having multiple operating system to boot any of them by just selecting it in the system setup.
Taking this into account, for example when you install Windows on a machine that already has Linux, Windows puts it bootloader to boot first, disabling the GRUB bootloader which is the standard bootloader in Linux. But thanks to this feature of selecting the bootloader from the system setup, one could re enable GRUB to have priority over the windows boot up.


These are the very basic concepts that are necessary to understand at a VERY high level how the Operating system booting works under UEFI.
Please let me know in the comments if there is a technical or concept error.