Below I will be explaining a lot of things that I just think needs to get out in the open. There are a lot of people that just don't understand the ins and outs of android and that go on faith that something is true right off the start, without any proof whatsoever. I hope to just explain some of this information a little better than it has been before.
This is generally a false statement but actually lies in the gray area between true and false. What people don't seem to realize is that android is designed to have a large number of tasks stored in memory at all times. Why? Well basically we are talking about a mobile device. On a mobile device things tend to be slower. The hardware isn't as robust as say a desktop or a laptop, so in order to get that same “snappy” feeling, there have to be workarounds.
One of these is how android deals with memory. Android will load up your apps and then keep them running until they absolutely HAVE to kill them. This is because that way, if you want to re-open an app, the system already has it loaded and can then just resume it instead of reloading it. This provides a significant performance increase.
What a lot of people don't realize as well is that android kernels have their own task manager. This means that:
There is only one case where having a task killer is a good idea, and that is when you want to kill ONE SPECIFIC APP. Killing all apps is never a good idea. You don't know what operations they are performing or if they are necessary.
This is a horrible idea. I will keep this short because it is simply just not a hard thing to understand. What is the benefit of having Xmb of FREE memory? Free memory is basically wasted real-estate. The worst part about it is that these scripts that set it are designed to tell the kernel-level task killer to begin killing apps when you have less than Xmb of memory free. Some people claim this is great because it keeps their task list clean, but it's a horrible idea and just leads to problems with redraws and instability. Android is built as an OS to run with its memory full at all times, because that allows it to run faster.
Ok, I'm sorry but it has to be said. Quadrant/Linpack/[insert random benchmark tool here] do NOT measure your phone's overall performance in a way that is meaninful, and benchmarks on mobile devices are basically nothing more than the placebo effect. Let me explain a great example of what I mean.
A perfect test of how arbitrary these numbers are is to try running quadrant/linpack under different types of strain. You will NEVER reproduce your exact results in any way meaningful. And you might say “well the numbers are generally the same if I run it over and over”. This may be true but let me then ask you this: What about if you run it while in a phone call? What about if you run it while playing music? What about if you run it while downloading apps from the market?
Sure you can get a 2000 quadrant score if you wipe out half of your apps and kill all of the processes running, but THAT'S NOT A BENCHMARK. Of course the numbers are going to be high when you do stuff like that, but would you actually USE the phone in that condition? A benchmark should be run under regular working conditions, not under some perfect conditions that are unreasonable and would never exist in the field.
So what do I mean by this. A GREAT example of how benchmarks are generally just a load of crap is Sapphire 1.0.0. In that release, I disabled stagefright media player by default. The thing is, stagefright decodes H.264-encoded video significantly faster (note: FASTER is not necessarily the same as BETTER) than opencore, which was the replacement. It also turns out that Quadrant bases some of its score on how fast you can decode said video format. The result? A stock Sapphire 1.0.0 release would score ~700 on Quadrant. I then released an update that would re-enable stagefright for those who wanted to, and the Quadrant scores jumped back up to ~1400. So what does this mean? Of the people who have installed that update, I have yet to hear anyone claim that their phone's overall performance has doubled, which is what the benchmark would suggest. Also, prior to re-enabling stagefright, the phone ran just as fast as after in the general case. The only difference was that opencore would take a few extra seconds to begin decoding H.264 whereas stagefright wouldn't. How this equates to a 50% reduction in speed is beyond me.
So what do I mean by this? Benchmarks are historically done to compare multiple systems that all claim to do the same thing. However, typically they would include a “base” or “gold standard” system in the mix as well. This base system would then be analyzed to determine how it performs under normal operating conditions, and then specify how it scored based on the methods of benchmarking that would be imposed. These scores would basically be seen as the “baseline” for the entire benchmark. The purpose for this was to then say that the systems being tested either did better or worse than what you would typically have standard anyway, and then from there measure which ones were the MOST better.
This is something glaringly absent from any of the benchmark programs out there. That is because it's a benchmark program. A benchmarking program is only aware of the system that it is running on. It may be able to compare the score of the system it ran on with other scores over the internet, but it still fails to explain what it is considering to be a baseline and what the scores even mean.
People who rant and rave about benchmark scores generally don't understand them or realize that they are mostly a lot of smoke and mirrors. I have NEVER used a benchmark to determine whether Sapphire was ready for release versus needing work, and the above reasons are why.
Also, these benchmark programs make a mockery of the whole idea of “benchmarking”. Performing a good benchmark is not done in 15-30 seconds by a program and the click of a button. A good benchmark involves hours of testing under multiple conditions as well as hours of determining testing methods and benchmark scoring rubrics. After all of this has been done, a good benchmark is explained in a lengthy article describing all of the work that went into the benchmark as well as all of the scoring rubrics, the test methods, and exactly what each part of the score means and why. It is then left open for others to examine the method and critique where the benchmark may be incorrect. This entire process is cut out in these so-called benchmark apps.
This is a hugely common thing for people to comment on, but generally has little to no bearing on source-based distros. Very few people actually rip apart new leaks and verify what has changed from build to build, but there are two important distinctions to make:
So what do I mean by this? Source based ROMs are those such as Sapphire or CyanogenMOD. We pull our source on a regular basis directly from AOSP (Android Open Source Project), and therefore anything we build from source will always be at least as up-to-date as the day it was built. There are a few exceptions.
There are a small number of files that are proprietary and therefore are obtained through leaks. The thing is that with the exception of a major release leak (i.e. the first eclair or froyo leaks), these files generally change little to at all. I personally looked at the FRG01B vs FRG22 leaks that recently came up, and so far as I could tell, only one file that source distros use changed, and it was a minor change to the IME Keyboard.
Needless to say, source distros do not generally care about intermittent leaks, but that doesn't mean that they are “behind the curve”.
Now on to hack-based ROMs such as Bugless Beast and stock leaks. These ROMs actually depend on these steady leaks to keep themselves up-to-date, as they have no way to upgrade the source that has been pushed to AOSP (since they don't build from source). In that respect, there are changes to AOSP reflected in these leaks that would make them very useful to these ROMs.
Thus, it's important to know that different builds are very different from each other, but for source distros it doesn't matter, whereas for hacked distros it does. If you can keep that in mind it will help you to understand whether or not your ROM is as up-to-date as it can be. :)