Mobile Foresight » Technologies

Xperia Arc’s low light camera – will Sony(Ericsson) ever come to their senses about the Mpixel race?

Jonas Lind — Fri, 28 Oct 2011 07:58:45 +0000

Xperia arc

I don’t usually pay much attention to Sony Ericsson’s products but last week I saw a billboard in the subway stating that the Xperia Arc sported a camera with excellent low light capabilities. A quick google search revealed that SE is pushing the 8 Mpix low light camera as a major selling point. The low light capabilities come from Sony’s new sensor technology Exmor R.

It looks like SE’s marketing department has realized that most of their customers put a high value on the ability to take “natural” photos in low light conditions without a disturbing flash. Back in 2009 I wrote a highly critical review of their then flagship model Satio. The Satio was equipped with an oversized 12 Mpixel camera that was mediocre in low light due to the small sensor size. Two years later and it looks like they’ve finally fixed the problem.

It’s great that Sony has developed the Exmor R sensor for improved low light photography. But if they want to exploit this technology and use it to jump ahead of the competition they should push the low light threshold to the extreme in a sensor with larger pixel size but fewer megapixels. Instead Sony has developed a small silly 16 Mpix sensor while most of their competitors are concentrating on factors other than megapixel count.

The Mpixel count is only one design parameter. Pixel size is as important as the number of pixels. If each pixel is larger it will capture more photons.

And the sensor is only one component in the camera. Compare it with the highly touted Nokia N8 camera. Nokia’s sensor is one the largest in any camera phone, the Zeiss lens is made of glass not plastic, it has a mechanical shutter, a Xenon flash, and a built-in ND filter to handle extremely bright shooting conditions.

It’s a shame that Sony(Ericsson) and the new owners Sony don’t understand that they could use this new technology for an extreme low light camera phone that would sweep the competitors away. If they developed a 6 Mpix Exmor R sensor with a larger sensor size and used a Zeiss lens made of hardened glass they would really have a winner on their hands.

New UI/UX after Apple’s iPhone/iPad

Jonas Lind — Fri, 09 Sep 2011 08:40:22 +0000

Sometimes I get the impression that the industry believes the iPhone and iPad represent the pinnacle of human technology. Even though the majority of the market attention is on these form factors, several new UI technologies are already out of the labs. These technologies have the potential to disrupt the traditional smartphone/tablet market and might pave the way for new types of products.

Here are a few examples that point toward a world after candybar multitouch. Exactly how they can be used and integrated in the UI/UX remains to be seen.

Demo of Microsoft Surface with PixelSense from Samsung

I have written about Microsoft Surface before, which is large horizontal multitouch screen built as a table. In the new slimmer version of Surface, Microsoft together with Samsung have developed PixelSense touch sensing technology. In PixelSense every pixel in the screen is also an infrared sensor that detects warm fingers on the surface. Just imagine what a future development of this technology could do if Samsung manages to fit the three RBG color sensors in every pixel. The surface could double as a copying machine. You put a paper, coupon or picture facing down on the surface, and when you lift it up, the copied object is displayed on the screen.

A technology for high performance multitouch screens has been developed by the Swedish startup Flatfrog. Their multitouch is based on an optical in-glass solution (Planar Scatter Detection) that also can be used to create multitouch on curved glass surfaces.

Another Swedish startup is Tobii, which has developed a technology for tracking eye movements. Using cameras that track the position of the pupil it is possible to calculate exactly what the user is focusing on. The company’s initial markets have been expensive high end systems for paralyzed people, market researchers, and academic researchers in cognitive psychology. Tobii has now begun to target the mainstream market together with Lenovo which are integrating eye tracking in a prototype laptop.

Kinect is a technology that Microsoft developed for their gaming console Xbox. It is an add-on gadget for your gaming console or flatscreen with facial recognition, voice recogniton and the ability to track gestures such as arm and hand movements. With Kinect you can control a game or PC by talking and waving your arms. It can be used for controlling an action figure or for moving between windows such as browsing your music collection, zooming in and out of a photo, etc. Up to six users can be tracked at the same time.

Even more futuristic UI/UX modalities are BCI technologies (Brain Computer Interface) where brain waves directly control an UI or some machinery. BCI has been used in research labs for a long time with electrodes implanted in the skull. Newer products based on less invasive methods with the electrodes attached to the scalp are now hitting the market, often in the form of a headset. The precision and bandwidth of these methods are still very primitive. One of the few things that can be reliably measured with BCI are emotive states such as relaxation vs. concentration.

Most of these new innovations are early in their life cycle and it is still too early to tell if anyone of them has a strong disruptive potential. New technologies drive development of new form factors. It remains to be seen if and how this will create future killer hardware. There is also a shortage of apps that can take advantage of the new features and turn them into compelling user experiences.

There are several hurdles to overcome. Products such as Kinect, Tobii and Surface put significant demands on processor capacity and there is a learning curve for any new UI technology. Prices have to come down for the large mainstream market to accept them.

I am slightly skeptical about a technology that requires you to wave your arms. What’s fine when gaming in your own living room, lifting and waving your arms for an extended period of time is tiresome. This has already been shown by users’ resistance to large vertical PC touchscreens.

It is possible that these new technologies will find their way into the candybar smartphone/tablet. But I think it is more likely that the future smartphone will integrate these new UI technologies without residing in the handset. If most tables, office desks, and bars are made of hard glass with MS Surface technology perhaps the user could just place their smartphone on the glass and have all their apps, contacts and pictures displayed. The surface might even have built in eye tracking. Or maybe Corning’s vision of a world of glass will come true and the nearest wall will be able to display your smartphone home screen with built in eye tracking for navigation in the wall. Just make sure to control your eyeballs – you never know who might be looking over your shoulder.

What will happen to webOS after HP’s exit?

Jonas Lind — Fri, 19 Aug 2011 12:21:12 +0000

Update: According to The Next Web’s internal sources, HP is NOT going to
sell or shut webOS down, just the underperforming hardware device division.
HP communicated this in a clumsy way which gave the erroneous impression
that webOS was dead or for sale. HP’s plan seems to be to license webOS to hardware partners. But if HP is going to sell their PC division, they can’t force
the buyer to promote webOS on the desktop platform. WebOS as a
competitor to Windows 8 still seems quite unlikely.

My previous blog post about the potential for HP/webOS to expand webOS into the desktop and compete with Windows 8 was only two days old when HP announced their plans to abandon the tablet/smartphone market and their commitment to webOS. I got it wrong. This has been an interesting week in a frantically fast moving industry. Friday is not over so there is still time for another major announcement. How about Microsoft buying Nokia or Dell buying RIM?

I think HP made a rushed and unwise decision. Their Palm smartphones and HP Touchpad tablet didn’t sell, but that was due to clumsy design, weak hardware, and bugs. It was not caused by some inherent weakness in webOS.

If they had persevered, promoted an ecosystem, and licensed webOS I think they would have had a chance on the market. After the Google/Moto deal several of the Android licensees (LG, HTC, Sony Ericsson, Samsung, etc.) would probably have been interested in an alternative OS to reduce their dependence of Google.

And as I said in my previous blog post, if and when the HTML5/cloud paradigm becomes dominant, most existing HTML5 web apps will automatically become part of a huge virtual webOS app store inventory.

I haven’t really had time to consider potential webOS buyers. Here are some rough ideas: If HP includes the patent portfolio in the deal they can probably get a better price. Google and Samsung might be interested for hoarding more patents. Other usual suspects would be LG, HTC, Sony Ericsson, ZTE and Huawei. Or possibly China Mobile, Verizon or another large operator. Amazon might be interested in using webOS for a tablet/cloud service offer. Dell, Lenovo, Acer or Asus might want to develop cloud-PC solutions based on webOS to escape Windows 8 and Microsoft’s stranglehold. Perhaps next week’s events and mega deals will provide some answers.

WebOS’s new life as HP’s Trojan horse

Jonas Lind — Wed, 17 Aug 2011 09:18:10 +0000

webOS from HP/Palm on different screens

The Google/Motorola deal suddenly makes it relevant to take a closer look at the other OS platforms on the market. Most industry observers in the mobile space have already written off HP/Palm’s webOS as a dead platform. From a smartphone perspective, this is obvious at first sight. With two percent of smartphone users in their only market (US), negligible global sales, and a thin app store, they don’t have much going for them. HP recently released a tablet powered by webOS but according to the reviewers the product is clumsy and behind the competition. There is talk about Samsung licensing webOS for a few handsets but no products have hit the shelves yet.

HP/webOS can not currently compete head to head with Apple and Android, even though the webOS technical platform is excellent. The platform is designed for true multitasking and for running web apps based on HTML5/JavaScript/CCS with hardware accelerated execution.

But there are a couple of scenarios where webOS can be viable. For example, if HTML5 becomes dominant and native apps fade into oblivion. In that case, the advantage of the market leaders’ huge app stores will become irrelevant and webOS will be able to compete on a level playing field. As webOS is optimized for HTML5 from the beginning, there is a good chance that apps will work better than on the competing platforms. If other OEMs become wary of using Android after Google’s Motorola purchase, webOS might be more attractive for licensing. An additional factor that could reinforce this scenario is the rise of cloud computing. If all this plays out, handset OEMs and mobile operators might consider webOS as a way of reducing the market power of Android, Microsoft, and Apple. But even if native apps are sidelined by HTML5, the smartphone market leaders will still have a huge advantage due to their brand, size, and market share.

However, if the game plan changes, webOS could reappear as a serious contender. If and when the cloud becomes the dominant computing paradigm, webOS will be a good fit for a range of devices. The drawback is that the mobile connection to the smartphone will always be unreliable and rather slow. If all your documents, contacts, messaging and apps are stored in the cloud this will be a problem. Enter the PC (and to some extent the tablet). For the PC one usually has a reliable broadband connection. That is where cloud computing (and hence webOS) would work best. And this is the direction HP is moving in with webOS. HP will install webOS on all their products (PCs, tablets, printers etc.) from 2012.

Industry observers predict that HP wants webOS so it can provide users with “experience roaming” across all devices, where webOS will ensure that users can keep their profile and contacts. The Synergy feature in webOS aggregates all the user’s contacts and supports that strategy. Enyo is webOS’s development framework and supports handling different screen sizes in a seamless way. This is probably part of HP’s plan.

But HP’s bolder move is to install webOS together with the upcoming Windows 8 on the traditional PC. I don’t know if this was part of HP’s original strategy when they bought Palm in April 2010 or added by the new CEO Léo Apotheker who joined HP in September 2010. If webOS boots faster than Windows 8, if HP gives the user a choice between webOS and Windows during boot, and if the user primarily uses the PC to access HTML5 web apps in the cloud – then perhaps users will choose to boot webOS instead of Windows and gradually stop using the legacy Windows platform. This would give HP huge leverage in future price negotiations with Microsoft. WebOS will be HP’s Trojan horse for breaking into Microsoft’s territory on the PC.

Windows 8 is a radical departure from the traditional desktop keyboard and mouse paradigm. Windows 8 comes with full touch screen support and their UI theme (“Metro”) has a look and feel with tiles that plagiarizes WP7 and most of the smartphone/tablets platforms. The traditional legacy desktop environment is hidden inside tiles. Windows 8 is built for app development in – surprise surprise – HTML5. The same as webOS. If we disregard the support for legacy desktop apps, Windows 8 and webOS will almost be head to head competitors. And while webOS is optimized for web apps, Windows 8 has to carry a huge legacy baggage to keep backward compatibility.

Will one of the most insignificant smartphone platforms rise from the ashes and conquer the giant of the PC era? Don’t rule it out just yet.

How cross-platform tools can end the OS wars

Jonas Lind — Mon, 04 Jul 2011 10:09:52 +0000

This article has previously been published on VisionMobile’s blog.

The Android vs. iOS vs. Windows Phone platform battle has been the talk of the industry for the last year. But the market share battle between handset platforms might not be as critical for the industry as many believe.

A popular view in the industry is that the market is inevitably moving towards an Apple-Google duopoly. Apple’s app store has more than 400,000 apps. Android is growing quickly from a base of more than 250,000 apps and is predicted to catch up with Apple later this year. Nearly 80 percent of all apps in app stores are controlled by these two market giants according to Distimo. Figures for Q1 2011 from Gartner show that the market share in the smartphone market for iOS and Android combined is 53 percent and rising.

But the duopoly may be challenged by the mobile web and cross-platform tools. HTML5 empowers all other platforms to offer apps through the browser. VisionMobile’s recent Developer Economics report shows that the mobile web (of which HTML5 is a subset) is already the third most popular platform in terms of developer mindshare after Android and iOS.

At the same time, HTML5 is overhyped and the belief that HTML5 will replace almost all native apps is in need of a reality check. Native apps will still offer richer functionality, better performance, and higher security compared to HTML5-based apps. A study by quirksmode.org has shown that every mobile WebKit implementation is slightly different, which could cause a problem for HTML5-based apps. In a recent whitepaper, Netbiscuits measured smartphone support for 18 features in HTML5 and showed that leading smartphones only offer partial (or no) support for a significant number of these features. Implementation is also fragmented. What works on iPhone will probably not work on RIM or Samsung handsets and vice versa. Or to quote Forrester’s take on the HTML5 vs. native debate: “The ‘Apps vs. Internet’ Debate Will Continue…to be irrelevant.”, “it’s not a question of ‘either/or’ when it comes to a choice between apps vs. the mobile Web, but both.”

The Landscape Of Cross-Platform Development Tools

The new types of cross-platform tools are more interesting than plain HTML5 because they can deliver higher performance and functionality than browser based HTML5. These tools produce apps as output and fall roughly into two categories:

1) Web apps/hybrid apps. These apps exploit the web engine (“web browser”) and are typically written in HTML/CSS/JavaScript.

2) Native apps. These apps are compiled into machine code and often written in C++ or similar languages.

Cross-platform tools are a nascent market with a flurry of startup activity over the last few years. The following diagram illustrates different trade-offs between complexity and performance in the cross-platform tools market.

Market segments for mobile cross-platform tools

Traditional websites: In the lower left corner is the traditional website, limited in performance but providing access to all platforms with no added complexity. Plain HTML5 could be included here once all browsers support the standard.

Web apps/hybrid apps: Adjacent in the diagram are HTML5 web apps that can be downloaded to the browser’s cache and run offline. They will offer better performance and only slightly higher complexity. One step up in the diagram is a market segment of cross-platform tools running simulated native. These tools deliver better performance but the complexity is also higher if the tool has to support multiple platforms. Here we find tools that produce web apps built on HTML5/CCS3 and JavaScript, with some added native elements, typically inside a native wrapper. These cross-platform tools often add native extensions that provide access to some low level native functionality. An example of a player in this market segment is PhoneGap, which is often used in tandem with the Sencha Touch framework. Other tools that run on top of PhoneGap are WorkLight and appMobi.

A closely related market segment is hybrid tools, where the HTML5/JavaScript input is translated into actual native source code. An example of a hybrid tool vendor is Appcelerator’s Titanium.

Other types of solutions which fall under the main heading of web/hybrid apps are based on Java, Lua, ActionScript or less common languages. The diagram shows how the heavily-fragmented Java ME offers inferior performance in spite of high complexity. The cross-platform tools Corona SDK and DragonRAD are based on Lua. Rhodes is based on HTML/Ruby while OpenPlug uses ActionScript (Flash) as source language. Kony uses drag-n-drop for building enterprise web apps. There is no reliable information about the performance/complexity trade-off for most of these solutions, so their exact position in the diagram above should be viewed as illustrative. In general, tools in which the resulting code is compiled or recompiled to native ARM machine code will have a higher performance.

Native apps: The second main category is native apps. In cross-platform tools for native apps, developers often work with a codebase in C/C++ or C# which is then semi-automatically ported to the target platform and device. Performance is significantly higher with native code, but so is the complexity. Players in this sector include Airplay, Qt and MoSync. The Airplay SDK (now Marmalade) originates in 3D gaming but can also be used as a general C++ cross-platform tool. Qt is a cross-platform UI framework that also can be used for native C++ porting. Qt primarily supports Nokia’s legacy platforms. MoSync is a cross-platform tool for general purpose C++ development, integrated with the Eclipse IDE and also available under an open source (GPL) license.

Cross-Platform Beyond Java – Native Extensions

The traditional approach to cross-platform development has been a lowest common denominator one – much like that taken by Java, Flash Lite and mobile HTML. This approach sacrifices performance, UI pizzazz and access to specific device features.

A workaround is to add native extensions. These can provide additional SDK/NDK libraries for the IDE and also give access to low level hardware functionality. Access to low-level hardware functionality can be managed by a device database that controls which conditional code will be executed on a given device.

Several of the cross-platform vendors have built such device databases with various levels of detail. A device database contains information on screen size, input modality and exact OS version, extending to detailed hardware configurations and known bugs with workarounds.

Using native extensions, it is possible to overcome the inherent limitations that plagued Java. Instead of “write once, run everywhere”, developers can spend 90 percent of their time developing a common codebase and 10 percent adding native tweaks and extensions for each platform and device. For software purists, the 90/10 solution might not seem very elegant, but it is a way forward that can handle the incredible complexity with thousands of devices running more than five OS platforms. In this way, app developers can manage one codebase and port it to target devices without losing functionality. In principle, using a (C++) cross-platform engine with extensions should be able to offer similar functionality with minimal performance penalty as compared to direct development for the target device. There will be significant economies of scale when the common codebase is tweaked for 100s of devices.

The Disruptive Potential Of Cross-Platform

There are few signs that platform fragmentation will disappear. It’s not just Android, iOS and Windows Phone 7, which are backed by corporate giants with deep pockets, but also smaller players like QNX (RIM), WebOS (HP), MeeGo (Intel, China Mobile) and Bada (Samsung). Add to that legacy platforms, which will be around for at least a few years: Windows Mobile, Blackberry OS, Symbian, BREW, Java ME and Flash. If we also include the main desktop platforms (Windows, Mac OS, Ubuntu), gaming consoles, set-top boxes, cars, and other gadgets, the number of platforms becomes unmanageable.

App developers whose clients need to reach the entire market, face the formidable task of supporting all platforms and devices. If they can use a cross-platform engine the productivity gains will be dramatic compared to paying for separate in-house dev teams for each platform.

Early adopters of cross-platform will most likely be large consumer businesses who need to target the mass market such as media companies, games houses, entertainment companies, banks, and any brand developing B2C apps. Similarly, government agencies are often required to provide non-discriminatory access to their services and cross-platform tools will enable them to do just that. Another group of early adopters of cross-platform tools is CIOs of larger corporations. They face increasing demand from senior staff who want to use their favorite smartphone for secure access of internal company data. Once these early adopters have driven down the prices and sorted out stability issues we should expect to see a fast uptake of cross-platform tools in the mainstream app development market.

Assuming more developers move to cross-platform tools, the power distribution in the mobile sector will be challenged. The difference in the number of available apps between dominant and up-n-coming platforms will be reduced. This will allow smaller platforms to compete on a level playing field.

Web apps and HTML5 should make the largest dent in the market power of traditional platforms. But the final nail in the coffin will come when C++ cross-platform engines can offer almost the same performance and functionality as coding directly on the target platform. This is possible if the cross-platform engines can fully integrate native platform and device extensions. In that case, developers of native apps might reconsider Android, iOS and WP7 and choose to code to a cross-platform IDE, not to the platform. In this scenario, the cross-platform IDEs would become players of equal or even greater importance than the native platforms. At the very least, today’s OS platform wars will move to a totally different level.

Comments can be left at VisionMobile’s blog.

The HTML5 hype – time for a reality check

Jonas Lind — Fri, 11 Mar 2011 09:32:30 +0000

The hype created by the promise of HTML5 has almost reached fever pitch during the last quarter. With HTML5/CSS3 it will be possible to run most types of applications directly in the “browser” and the need to install apps that execute native code will be a thing of the past. HTML5 can run from a cache in your smartphone/tablet/PC even if you are offline and the app can access the phone’s GPS, compass, accelerometer, touch recognition and native video/audio control. App developers will no longer need to develop separate native versions for iPhone, Android, WP, Blackberry, Samsung/Bada, and WebOS. Just write once in HTML5 and run everywhere.

There is truth in all this and HTML5 is a great technology. But as usual during the peak of inflated expectations people tend to forget the limitations. HTML5 is still an immature technology. The final draft will be finished in mid-2011 and the W3C recently stated that the formal standard decision will be delayed until 2014. When people actually start using HTML5 the experience will most likely be underwhelming as developers are faced with the limitations of the technology. (This view is supported by comments from industry conferences.) Older handsets will most likely not be able to run full HTML5 web apps, which kind of defeats the vision of universal access, at least for the near future.

Native apps will always offer better performance, better UI/UX, and better integration with the device hardware. For example, HTML5 does not support augmented reality. HTML5 will, over time, be able to close the gap but if we assume that the native app platforms continue to develop, the goalpost will be moving as well. Ecosystem owners (Apple, Google, etc.) will of course work to make their native development environment as pleasant as possible to work with. In addition, cross-platform tools in the native environment will reduce the effort of porting from one platform to the other.

What the market tends to forget is that the fundamental trade-off between standardization and flexibility will not go away. By complying with the HTML5 standard, handset makers and web app developers will be unable to differentiate outside the limits set by the standard. It is inevitable that one global standard will not be fully capable of adapting to a highly heterogeneous base of various screen sizes, handsets, tablets, etc. Once a committee-based standard is finalized, innovations and new product features that are introduced after that point will not be included until the next upgrade of the standard. Apps will be better at taking full advantage of device variation and new functionality.

I doubt vendors will be able to resist the siren song of differentiation. When they give in to this temptation, the evil twin of differentiation – fragmentation – will rear its ugly head. This fragmentation will either undermine HTML5 as a universal standard (which will make it less attractive in the same way as Java ME), or be expressed in the form of more native apps.

Another fundamental trade-off in software is between raw performance and developer convenience. Coding in high level languages is easier for less experienced developers and the pool of HTML/JavaScript developers is much larger than the number of experienced C++ developers. It will be cheaper and easier to develop in HTML5 but performance will have to be sacrificed. For less advanced applications this might be a good trade-off but it is a trade-off nonetheless. Efficient low level coding also translates into lower battery drainage, which is important for smartphones.

In many cases it will be better to have a HTML5 web app than no app at all. But if high performance is critical, native apps will be the obvious choice. Performance is not just relevant in obvious areas such as games. If users expect touch-based smartphone apps that don’t freeze when browsing or scrolling it might be a critical competitive differentiator for all apps.

As pointed out by Forrester, this entire “either/or” scenario with HTML5 vs. apps is driven by vendor politics. Although I think that another strong driver is the large community of web developers who find C programming too hard. These web developers will gladly embrace HTML5 as a way to enter the mobile marketspace.

HTML5 is pushed by Google, Microsoft, Facebook and Apple. Continued browser dominance is critical for Google’s ad business. Microsoft and Apple want to kill Flash. Facebook wants universal access. But actions speak louder than words; Google is actively recruiting app developers and releasing more of their own services as apps. For example, Google Voice, Google Places, and now also Google Translate. (Google Earth is the most widely known example despite it having been around for years.) Today large players that quickly want to reach the market with their services are developing native apps when the functionality demands it, in spite of the pro-HTML5 rhetoric.

Amazing ways to interface with your computer

Jonas Lind — Thu, 17 Dec 2009 11:24:27 +0000

Eventually the mouse and keyboard will be replaced as our primary way of interfacing with the digital world by something else. Voice input, and pen/finger based touch screens are the most well-known candidates but there are several other more advanced technologies being developed.

One example is I-Tech’s Virtual Keyboard (introduced in 2005) with a red laser that projects a keyboard on to any surface. You type and the sensors will detect the key that got the light blocked. The $170 price tag has prevented this product from entering the mainstream market.

Virtual Laser Keyboard

Another example is multi-touch screen technology. An amazing demo at TED from 2006 shows envisioned ways of using it on a high end computer with a large screen. A more simple version of multi-touch has been included in the iPhone since 2007.

The coffee table sized Microsoft Surface is a high end multi-touch product that has been on the market since 2008. The horizontal screen can identify what objects you place on the surface. If you put your smartphone on the screen and drag pictures to the mobile with your finger they will be uploaded to the phone (and vice versa). As long as the price tag stays at $13,500 the market will be limited to casinos, hotels, and eye catching marketing events. Visually compelling video demos from Microsoft are here and here and another from Popular Mechanics is here.

Microsoft Surface: multi-touch flat screen

Competing products are the DiamondTouch Table from Mitsubishi, the Malaysian SmartSurface, and the iTable from PQ Labs. A less advanced product with multi-touch is the high end PC model TouchSmart from HP. In the six figure price range the Multi-Touch Collaboration Wall from Perceptive Pixel has customers such as the U.S. military and CNN (they used it in their studio during the presidential election).

The Multi-Touch Collaboration Wall

These new technologies are amazing and should inspire creative thought about new and unexpected ways of using them. You might find some demo applications silly but the point is to demonstrate the opportunities. It’s up to you, me, and others to discover the killer apps for this technology.

However, the most mind-boggling prototype has been developed at MIT Media Lab by the genius inventor Pranav Mistry in his project SixthSense.

He has a similar vision of blurring the line between the digital world and our physical surroundings and making it possible to access and interact with computers without dedicated input/output interfaces. He uses standard products, and assembles them into a seamless experience with a wearable system connected to your mobile. He uses a mini projector to display text and images on any surface and a camera to scan your hand gestures and objects in front of you. The software for the system will be released to open source developers any day now.

Sit back and enjoy this fantastic presentation from TED India.

Flawed design in Sony Ericsson’s Satio camera phone

Jonas Lind — Sun, 23 Aug 2009 23:16:08 +0000

Satio smartphone with camera

All mobile phone vendors hope to repeat the success of Apple with a new Jesus phone. Sony Ericsson’s candidate is Satio (previous working name Idou) and they are positioning the integrated 12 megapixel camera as the “killer app” that will lift Satio above the other smartphone competitors when it hits the shelves this fall.

Unfortunately, I think they have made a serious design mistake in their choice of camera. For marketers that are used to the IT industry’s logic that “more is always better” it is a given that cramming as many pixels as possible into the smartphone is a desirable goal. Sony Ericsson’s motivation is that the Facebook generation wants an easy way to take high quality pictures. That is correct, but a 12 MP camera phone is not the answer.

The problem is that a sensor with a lot of megapixels diminishes the ability to take good pictures in low light which, for most users, is much more important than taking high resolution pictures with 4000 x 3000 pixels. Sony Ericsson (SEMC) will run into problems when disappointed buyers realize how limited the camera is in practical usage.

12 megapixels used in the tiny sensors in a camera phone is not the same as 12 MP used in larger DSLR cameras. The number of megapixels is not the only performance factor. How tightly each pixel is packed on the sensor (Mpix/cm²) is of equal importance. If too many pixels are crammed on a sensor that is less than a square centimeter (6 × 4.5 mm if Sony’s new sensor is used in the Satio) each pixel will be so tiny that the physical limitations of the number of photons that can hit this pixel will determine the capability to produce an image.

If the camera sensor is viewed as a football field filled with buckets, the photons can be viewed as a rain of billiard balls that fall down in the buckets. Each bucket is one pixel. Stronger light produces many billiard balls which increases the precision when the number of balls in each bucket is counted. Weak lighting conditions (few billiard balls falling down) might work if each bucket is large enough to at least catch a few balls. But if the same number of buckets is crammed into a handball field each bucket will only be the size of drinking glass and the errors (noise) will be much larger because the billiard balls are too few to fill up all the small glasses.

For pictures taken in direct sunlight in the middle of the day or with a strong flash, the lowest sensitivity (ISO 100) is sufficient. Under these lighting conditions a small sensor (the handball field) is almost on par with a large sensor (the football field). There are still differences in quality because the optics in a small sensor camera is always inferior to larger, high quality optics. Quality differences are also caused by the fact that the leading camera vendors (Nikon and Canon) have more experience with electronic image processing than new players such as SEMC.

But as soon as you take pictures in low light and have to increase sensor sensitivity to ISO 400, 800 (or even higher) the difference between a large and a small sensor with the same megapixel count becomes dramatic.

In web forums such as Nikonians and review sites like DPReview most pro photographers and photo nerds have been in agreement about this for a long time. They are generally skeptical about small point-and-shoot cameras since the image noise level becomes unacceptable indoors or with low lighting. They are also critical about the way the vendors try to compensate for mediocre image quality with exaggerated electronic post-processing, by saturating the colors and by increasing edge sharpness. The result is quite often pictures that look unnatural. The table below shows the differences in the sensor area and pixel density for a selection of digital cameras and camera phones.

Sensor format and pixel density

	Mpix	Sensor area (mm²)	Pixel density (Mpix/cm²)
System cameras
Canon EOS 50D	15	22.3 × 14.9 = 332.3	4.5
Nikon D90	12	23.7 × 15.7 = 372	3.2

System cameras with full-frame sensor
Canon EOS 5D Mark II	21	36 × 24 = 864	2.4
Nikon D700	12	36 × 23.9 = 860	1.4

Compact cameras
Nikon Coolpix P6000	13.5	7.40 × 5.55 = 41.1	32.8
Olympus Stylus 9000	12	6.13 × 4.60 = 28.2	42.6
Samsung HZ10W	10	6.13 × 4.60 = 28.2	35.4
Fuji FinePix F31fd	6	7.60 × 5.70 = 43.3	13.9

Camera phones
Sony Ericsson Satio	12	6 × 4.5 = 27	44
Sony Ericsson K850	5	5.22 × 3.91 = 20.4	24.5
Nokia N95	5	4.86 × 3.6 = 17.28	29

Notice in the table above that the value for pixel density (megapixel per square centimeter) should be as low as possible. That figure is a measure of how densely the pixels are packed on the sensor and it is of almost equal importance as the number of megapixels. With a sensor area of 27 mm² Satio will have a sensor in the same range as the smallest point-and-shoot cameras. This will give Satio a pixel density of 44 megapixels per square centimeter which is very high.

Another observation from the table is that there are large differences in the sensor area between the camera types. The sensor in most DSLRs is around 350 mm². Compare this with the sensor in compact cameras which is between 28 and 40 mm². The sensor in a DSLR has an area that is twelve times larger than the smallest compact digital camera, thus providing twelve times higher capacity to collect light.

The first vendor to use a small sensor with fewer megapixels which is optimized for maximum low light performance will capture an empty market segment. There are no products on the market today to fill this latent demand. A few years ago Fuji released the Finepix F31fd camera, a 6 Mpix compact camera using a larger sensor with very good low light performance. It was even possible to take decent night pictures without a flash at ISO 1600 which no other compact camera had ever managed. When Fuji discontinued the model the camera gained cult status and the used prices on eBay have sometimes been close to double the new price.

Of course a company should not design their products for the nerd market. However, it is worthwhile to listen to advanced users. In this case they are right, high sensitivity in low light is relevant for everyone. Taking pictures indoors of your friends without being forced to use flash is perfect for the Facebook generation.

That the mass market customer believes “the more megapixels a camera has the better it is” is natural given that almost all marketing from the vendors has focused on megapixels. However, sooner or later the well-founded criticism of the megapixel obsession (e.g. from the New York Times’ technology editor) will bring awareness to the general public.

The only major advantage with a high megapixel sensor in a camera phone is that you can zoom in electronically without a disastrous loss of resolution. The electronic zoom only provides magnification from the center of the sensor if the lens has no moving parts. However, the marketers’ obsession with large zoom range (12x for Satio) is also criticized by those who actually use these cameras. Even if a camera has image stabilization is not enough to compensate for the fact that light sensitivity and resolution deteriorate significantly with electronic zooming. The New York Times points out that a good wide angle range is more important (so you can take pictures of all your friends around the dinner table) than useless telezooms.

A manufacturer that launches a camera (or cam phone) with a smaller zoom range and superior low light performance but with fewer megapixels can easily communicate user value. A marketing message could be built on comparing test pictures or copy text such as: “No flash at the wedding”, “Say goodbye to washed out flash images”, “Take pictures without anyone noticing”, etc.

If Sony Ericsson wants to reduce the pixel density my suggestion is that they develop a much more light sensitive sensor of the same size with a resolution of 2400 × 1800 pixels. This resolution will deliver 4.3 MPix pictures and the pixel density will be 15.9 Mpix/cm² which is much better than 44. A resolution of 2400 × 1800 pixels should be comparable to the resolution of a typical PC screen. Most screens have a lower resolution of 1680 × 1050 or 1920 × 1080, which means that the picture’s size has to be reduced in order to fit on the screen. Even larger 24 inch and 27 inch screens don’t have a higher resolution than 1920 × 1200. (Since most digital pictures are only displayed on a screen and never printed on paper, the higher resolution requirements for printing are not as relevant here.)

Sony Ericsson’s Satio as a case study

For Sony Ericsson’s sake one has to hope that the Satio will be well received by the market and that all the other advanced features are enough to convince the customers, in spite of the problems with the camera, to buy the smartphone. SEMC’s choice of camera is of general interest for product strategists in the tech sector and I will use it as case for an analysis of how the process went wrong (from my perspective as an outsider) and give some free advice to SEMC:

Do not allow internal politics within the Sony Ericsson alliance to govern product development. If Sony was given the responsibility of product development for the camera just because they have their own camera division it was mistake. As a camera maker Sony is weak brand. For instance, their DSLR division is built on Sony’s 2006 acquisition of two budget brands (Minolta/Konica) that were too weak to survive the competition from Nikon and Canon. Just because Sony developed a 12 Mpix sensor that they adapted for camera phones does not mean that SEMC should be forced to use that sensor.

That SEMC did not pay attention to photographers’ discussions on forums and in product reviews is a sign of weak competitive intelligence and consumer insight. Criticism of the megapixel race and clumsy image post-processing is readily available on the Internet. Even though SEMC has no intention of developing cameras they need to be competent buyers of the camera module.

SEMC (or another vendor) should be brave enough to go against the group think in the industry. Unfortunately, most consumer camera manufacturers have been technology driven in the wrong way, for example in their obsession with zoom range and megapixels. (My impression is that SEMC is stuck in the same pattern. A few years ago I spoke with one of the senior executives at a conference about the downsides of the megapixel race. My concerns were completely dismissed.) In addition, the vendors have also had a Microsoft-style ambition to fill their products with 100s of unnecessary, silly functions such as smile detection. At the same time they have neglected to implement basic functions such as lossless rotation of JPG pictures. (The last time I checked this in 2007, several compact cameras degraded image quality if they were rotated 90 degrees.) By breaking conventions and getting it right it is possible to gain a strong competitive advantage. Just look at the iPhone.

SEMC should have procured technologies from leading vendors with strong brands. Nokia uses Zeiss optics in their high end camera phones, which improves the image quality somewhat. SEMC could have done the same or partnered with Leica. In the built-in camera module Sony sells as an add-on to their new 12 MP sensor the lens is made of plastic. Plastic!

Using the same argument, SEMC should have bought the software for noise reduction, color balance etc. from Nikon or Phase One (though Nikon would probably refuse to sell their technology to a company affiliated with Sony).

SEMC should already prepare for an immediate launch of the next model ”Satio II” this fall if Satio fails on the market. It should use a 4.3 Mpix sensor designed for low light and use a smaller zoom range (3x).

This article has previously been published on my Swedish blog.

The telcos’ dilemma: LTE or HSPA+ (deja vu 2002)

Jonas Lind — Tue, 11 Aug 2009 13:00:48 +0000

Will financially burdened telco operators postpone investments in the 4G technology LTE? This has been an ongoing topic within the industry for over a year. LTE provides the operators with higher capacity, spectral efficiency, and data rates but requires heavy investments in new infrastructure. The alternative is to settle for the 3G upgrade HSPA or to upgrade the installed networks with HSPA+ (“turbo 3G”). An investment in HSPA+ is both a faster and cheaper solution which can work for several years unless the traffic soars too fast.

Due to the economic crisis there have been speculations that almost all operators will choose the incremental upgrade HSPA+ to save money and wait with LTE until the crisis is over. In that case, LTE deployment would not begin until some time after 2012-2013. Such a scenario would put the market leader Ericsson in a difficult situation if expected revenues are delayed several years. In addition, the competing technology WiMax might be able to establish a foothold if LTE is late to market.

This development will most likely not take place. Some operators have a different strategic position and for them HSPA+ is not an alternative. They are already investing in LTE and their networks are scheduled for operation in 2010. These early customers give the vendors the opportunity to get volume production started, test their products “live”, and stabilize the technology.

China Mobile is stuck in a local 3G standard (TD-SCDMA) that has failed to compete with WCDMA and CDMA2000. By investing in LTE, China Mobile can begin to migrate their customer base to a global standard where economics of scale and fiercer competition between the vendors will result in lower prices. The same logic applies to Verizon Wireless which is investing in LTE in order to move away from the peripheral standard Cdma/EVDO faster. NTT DoCoMo is investing in LTE networks and follows its tradition of early upgrades to new technologies (compare with DoCoMo’s early upgrades to WCDMA after the millennium). It is no surprise that the Scandinavian rivals TeliaSonera and Telenor/Tele2 also have made the decision to build LTE networks.

There are interesting parallels between the operators’ dilemma today and their investment choice around 2001 – 2002. At that time their decision was between a cheap upgrade of the GSM networks to GPRS/EDGE or heavy investments in 3G (if we disregard the spectrum auctions that forced the operators to invest in 3G). Today the choice is between HSPA+ and new LTE networks.

In hindsight the notion that investments in GPRS/EDGE and 3G were mutually exclusive was exaggerated. Directly after the major 3G decisions were made, the EDGE technology was almost considered to be outdated. But when data traffic increased the operators that had begun to deploy 3G networks upgraded their old 2G networks anyway. As the EDGE technology matured and prices fell it became a simple business decision to upgrade.

By the end of 2008 more than 80 percent of all GPRS networks had been upgraded to EDGE with few paying any attention to this. Today there are additional upgrades planned with EDGE Evolution, which will deliver data rates of 1.2 and 0,474 Mbit/s in the down-link and up-link respectively and an improvement of spectral efficiency and latency by a factor of two.

The total global market size makes it possible to support and develop several overlapping technology standards simultaneously. (There is even development of plain GSM, now with the innovation orthogonal sub channel from Nokia Siemens, which doubles voice capacity.)

The categorization of technologies such as 2G, 3G and 4G becomes increasingly irrelevant with a market where competitive standards from both 2G (EDGE), 3G (HSPA) and 4G (LTE) are developed and upgraded simultaneously. The choice between HPSA+ and LTE doesn’t have to be an either or, it can be both. My estimate is that the operators that are currently investing in LTE will eventually upgrade their old 3G networks to HPSA+ when the technology has matured and the prices have decreased.

This article has previously been published on my Swedish blog.

Why did the vision for public W-Lan networks fail?

Jonas Lind — Fri, 26 Jun 2009 22:51:40 +0000

Today it is obvious that business models that are based on public Wlan networks will never live up to the hype from 2002-2004. The dramatic price pressure for mobile data via 3G/HSPA undermined the market for Wlan operators. The choice is easy between spotty Wlan coverage and comprehensive geographical service with a 3G dongle.

Other players within the industry can learn several important lessons from the relative failure of this business model. If we go back to 1999-2002 we see that there was enthusiastic interest in Wlan and the possibility to create a disruptive innovation that could marginalize the traditional mobile operators.

The core of this hype was a simple analysis where the capacity/cost ratio for a Wlan access point was compared to a large 3G base station. A Wlan base station cost over 1,000 Euros and had the ability to transfer 5.5 Mbit per second. A large 3G mast cost 100,000 Euros and had a capacity of 4.5 Mbit per second. This capacity had to be shared by all the users within the mast’s area of coverage which meant that two surfers who each streamed 2.25 Mbit per second consumed the base station’s entire capacity. The cost advantage was dramatic. 3G was 100 times (10,000%) more expensive.

However, this is an oversimplified calculation: 3G micro base stations do not cost 100,000 Euros. When a 3G network is purchased there is built-in support for functionality such as roaming, handover, radio planning, access control, billing, etc. Wlan operators have to pay to develop solutions for this when their networks expand and their customer base grows.

Another factor is that Wlan base stations only cover an area within a radius of 30 (indoors) to 140 meters (outdoors). Geographical Wlan coverage requires many access points. But even if one adds the costs for the core functionality of the network as well as the cost of deploying a large number of Wlan access points, the cost advantage in favor of Wlan remained (even though the cost advantage was significantly lower than a factor of 100). In addition, Wlan networks have a higher capacity per square meter than 3G.

That’s how things stood seven years ago. Wlan was popular among the IT community, academics and VC investors. Wlan was embraced in a similar way as Linux and Open Source. It was only telecom operators and Ericsson/Nokia/Siemens who were upset when everyone criticized their 3G plans and the sky high prices they paid at the 3G auctions. I have to admit that I was also a Wlan enthusiast at that time.

What happened thereafter was that the competitors (the telecom operators and Ericsson et al.) responded to the threat, development took an unexpected turn and the Wlan operators shot themselves in the foot.

If one read between the lines about how Ericsson and the others in 3GPP reasoned before the Wlan threat became serious, the vendors’ plan was to begin by selling basic 3G networks at full price. The next phase was to sell capacity upgrades over several years at full price and finally begin marketing 4G (“LTE”) well after 2010.

When the Wlan threat appeared the telecom vendors were forced to rush their product development as well as market their upgrades such as HSPA earlier than they had planned. When the indebted operators were to be convinced to order 3G networks during the difficult period from 2002-2004, price pressure resulted in even lower prices. All of this eroded parts of the Wlan operators’ cost and capacity advantage.

Another factor that unexpectedly contributed to the undermining of the Wlan business model was the unused capacity in the finished 3G networks after 2006. The customers slow transition to 3G and the problems for pure 3G operators (such as Hutchisons 3) to acquire customers lead to desperate operators who were stuck with newly built networks without enough traffic. When 3 began to market fixed price 3G dongles they redefined the market with an offer that many other mobile operators were forced to copy. The 3G dongle became stiff competition for Wlan. Lower bandwidth, but much better geographical coverage.

It was actually the Wlan operators themselves who sabotaged their own market. By continuing to have unreasonable prices and not offering attractive roaming packages they missed the window of opportunity that existed before the 3G networks were fully up and running.

What originally made the concept of Wlan so exciting was the vision of a small-scale network deployment where the initial costs were minimal. If all of these networks had worked together and used one clearing house that was in charge of roaming, the users could be offered a large aggregated “network” with good coverage. Unfortunately, this did not take place.

The prices were set far too high and the operators did not ensure that the roaming contracts were reasonably priced. Considering the spotty coverage and the fact that few Wlan networks provided overlapping coverage, the operators should of course have created a single roaming pool. But they didn’t. They didn’t even understand that they should take advantage of the market for pre-paid cash cards. The prices were set for business users who were forced to access the service and login was made unnecessarily difficult.

Despite the fact that the Wlan industry was unable to live up to its promises and it will probably prove to be a wasted investment, we should all be grateful for the attention it has received over the last nine years. Wlan (and its cousin WiMax) have put pressure on Ericsson and the traditional telecom operators. This has increased the pace of innovation. The threat from Wlan operators will prevent greedy telcos from inflating their prices. The total market for all types of Wlan products has grown thanks to Wlan operators, and this has pushed the prices down.

This article has previously been published on my Swedish blog.