News feeds
Mini Map - A Web page visualization method for mobile phones
- , ,
- , ,
- , ,
- , ,
Abstract
When designing a Web browser for a handheld device, one of the most important problems to be solved is finding means to allow users to efficiently view and navigate arbitrary Web pages, most of which have been authored for large, desktop-size, screens. Unlike the current „state of the art” approaches, which reformat the entire Web page into one long column, thereby eliminating the need for horizontal scrolling, our solution aims to preserve the original look and feel of the page. We have developed a method, called Mini Map, comprising a set of modifications to the browser’s visual formatting model as well as a set of user interface controls that let the user recognize the structure of the Web page and quickly navigate to the interesting content. Our method is implemented on top of WebCore and JavaScriptCore, components of Apple’s Safari Web Kit, originally developed by the KDE community, which were ported to the S60 platform. The paper explains the general architecture of the browser, followed by a description of the main features of our method, and discusses our plans to work with the Open Source community.
Introduction
As large colour screens, faster wireless networks, and lower data prices become more commonplace, accessing the full Web, besides mobile only content, from your mobile phone becomes a reality. A study we conducted in Germany, UK and Singapore revealed that 63% of smart phone data traffic originates from browsing [Fel05]. However, capabilities of modern mobile Web browser have not been meeting end-user needs, mostly for two reasons. First, most mobile browser engines are not able to render full sized Web pages because they were designed to render mobile content in WML [Wap98] or XHTML MP [Oma01]. Second, most mobile browsers that are capable of rending full Web pages re-format them into a long content column both difficult for end-user to use and content developers to create content for. In this paper we present our implementation of a new Open Source based mobile Web browser for S60, an easy method to view full Web pages called Mini Map, and discuss our plans to work with the Open Source community.
Related work
The current state-of-the-art approach in Web content rendering for hand-held devices is called Narrow Layout. Narrow Layout re-formats the Web page into one column that fits the width of a typically small handheld device display. The goal of the method is to eliminate the need for horizontal scrolling and let the user access the content with vertical scrolling only. One example of the Narrow Layout method is Small-Screen Rendering [Lie05]. Usability studies [Rot03] and our own experience have shown that Narrow Layout suffers from significant limitations: it often destroys the logical ordering of the content, thus preventing the user from recognizing familiar pages; the first screens of the pages within the same Web site may look exactly the same and be inextricable; and pages that rely on a two-dimensional layout (e.g. timetables, maps) can break easily.
In addition to Narrow Layout, there exist several Web viewing methods for handheld devices that apply the Overview plus Detail method [Bau04, Buy02, Hoi03, Ful01, Lam05, Mil02, and Wob02]. In this method, an overview is used to display the whole data, while a detailed view shows a close-up portion of the data. Other Web viewing methods simply eliminate parts of the content [Gup03, Wob02].
Nokia S60 browser architecure
In order to be able to prototype and experiment with a new Web page visualization method, we first needed a browser engine capable of displaying full Web content and not just XHTML MP or WML. Since we needed access to the actual engine source code, we decided to use Open Source components. Furthermore, we determined that Open Source Software is also by far the best approach for building a quality product, saving development costs compared to building our own full Web browser from scratch, or to licensing a commercial 3rd party browser.
We therefore decided to use the KHTML library of the KDE Konqueror browser [Kde06]. The main reasons for this choice relied on the clean design of KHTML, its small code size, good start-up performance, low memory consumption and sufficient Web site compatibility. We found our decision reinforced when Apple announced in January 2003 that they had made the same choice for their Safari browser [App06]. We decided to start with Apple’s code base, WebCore and JavaScriptCore, because we benefited from Apple’s changes to the original KDE code to make WebCore and JavaScriptCore more portable and from their improvements to performance and Web site compatibility.
The general architecture of the new S60 browser [S60] is depicted in architecture.png.
Figure 1: Nokia S60 browser architecture
Open Source components
WebCore is an Open Source, cross-platform library that contains two major components: KHTML and KWQ. KHTML encapsulates the functionality required to parse, format and render HTML documents. Its most important parts are the HTML and CSS parsers, the HTML and XML DOM (Document Object Model) implementation, the document layout and rendering logic as well as main memory cache and a resource loading component. KWQ is an adaptation layer for KHTML that provides access to Symbian operating system (OS) specific functions (see Section 3.2).
JavaScriptCore is an Open Source, cross-platform JavaScript engine. JavaScriptCore is integrated to the DOM implementation in WebCore to access the HTML document and pass events.
Certain performance optimizations were needed to ensure a suitable user experience, given the particularities of the mobile device environment. For instance, one of the most important performance bottlenecks is caused by the wireless network latency. Since JavaScript typically needs to be executed “in-place”, the mark-up parser will block every time an external script is encountered. This can sometimes significantly delay the rendering of the document, resulting in poor user experience. Similar delays can be caused by the need to load external style sheets, so the solution to this problem was to enable a special “fast rendering” mode, where the document is initially rendered with partial styling information and without executing the external scripts. To speed up the download of these resources, some changes had to be made to the WebCore Loader component, so that the other types of content (e.g. images) are fetched with lower priority. When the scripts and styles are finally downloaded, the document is re-parsed and rendered in its final form. The WebCore caching policy was also modified in order to allow these resources to be cached more aggressively than images.
S60-specific components
The S60 WebKit library contains platform dependent functionality such as image decoding, graphics drawing, resource loading over Symbian OS HTTP stack or from the device local storage, etc. The WebKit library also implements a “browser control interface” that allows S60 GUI applications to include the browser as a normal control. WebCore and WebKit interact following a bridge-like pattern, where certain components defined in WebCore delegate operations to WebKit objects, which are part of a different class hierarchy.
An important component of the browser adaptation to Symbian OS is the Memory Manager. The motivation for the existence of this component is twofold: first, due to the limited available amount of RAM in a typical mobile phone and lack of virtual memory in Symbian OS, out-of-memory (OOM) situations are quite likely to occur and should be handled in a graceful manner rather than letting the application crash. Secondly, the browser engine deals with many small objects and the existing Symbian OS memory allocation routines are inefficient. The Memory Manager addresses these problems by providing ways for an application to handle OOM situations as early as possible and by overloading the system memory allocation routines with more efficient ones.
Mini map method
The most important problems with current state-of-the-art Narrow Layout Web page viewing methods, as outlined in Section 2, can be summarized as follows: the user cannot easily navigate to the desired content in the reformatted one-column page, since it is impossible to tell exactly where this content will be placed. The only way to reach it is to scroll down while paying very close attention to everything that appears in the viewport. When following a link that targets a page inside the same Web site, it will sometimes be very hard for the users to see that the new page was actually loaded. This is because many Web sites have pages that share a common header (usually a navigation pane). This header often ends up filling the entire device viewport, hence the first view of any page on the Web site will look identical on a small screen. For a more extensive discussion of the drawbacks of Narrow Layout methods, see Section 2 in [Rot06].
To address these types of problems, we followed a different approach. The Nokia S60 browser renders each Web page as intended by the author, respecting, with one exception (see Section 4.1), the CSS layout definition. The result is that the Web pages retain their original look and feel and thus become recognizable for a user who is familiar with them from a desktop browser. To ease navigation, we provide additional UI mechanisms such as the Mini Map, the Page Overview and the Visual History.
Narrow text columns
Rendering a page in its original layout, however, can result in having text paragraphs that are wider than the viewport. The user is then forced to scroll horizontally in order to be able to read each line of text. To avoid this situation we have made one modification to the CSS layout algorithm, by forcing the text runs to break whenever they exceed the viewport width. Another optimization, enforced only when the screen resolution is below a certain threshold, consists in formatting the page so that the text appears larger relative to the rest of the visual contents.
Mini Map
To give the user a sense of orientation and overview of the Web page, the browser provides a “Mini Map” (minimap.png), which is a thumbnail overview of the page (or of parts of it, if it is too large).
Figure 2: Mini Map
The Mini Map is rendered translucently over the main content in the viewport and appears automatically whenever the user continuously scrolls the page. A red rectangle in the Mini Map indicates the currently visible part of the Web page in the main view. This is actually the most prominent feature of our method, hence the name “Mini Map Web page visualization method”. The rest of the features described in Sections 4.3 and 4.4 are built having the Mini Map as the starting point.
Page Overview
The Page Overview provides a full-screen overview of the Web page. While the Mini Map becomes visible automatically, Page Overview needs to be activated and de-activated manually. The interaction model is otherwise identical.
From our experience, Page Overview and Mini Map are complementary features. Our studies have shown that it is easier for the users to discover the Mini Map (since it appears automatically), whereas Page Overview, due to its full-screen size, provides more context. We also found that the Page Overview works better on familiar Web pages, while the Mini Map is more suitable for finding content on pages that are visited for the first time. A screenshot with this feature can be seen in page_overview.png.
Figure 3: Page Overview
Visual History
Visual History is our improved version of the usual "Back" and "Forward" functions found in any Web browser. It displays the thumbnails of previously visited Web pages and allows the users to quickly jump any number of steps in the page history. This way, this UI control effectively eliminates the need to wait for intermediate pages in the history to render. The Visual History and the Mini Map have been the two features that created the biggest "wow" effect in user tests and when demonstrating the browser. page_overview.png depicts the Visual History.
Figure 4: Visual History
Open Source plans
Nokia is committed to Open Source, and we actively plan to contribute our mobility expertise to further develop the browser in collaboration with the Open Source community. Our modified source code for WebCore and JavaScriptCore is available at the Nokia Open Source Browser Web site. In order to give the community access to a complete mobile browser for S60 we consider to release our S60 WebKit source code and tools, including components such as S60 WebKit, Memory Manager, and a sample UI. The Nokia browser user interface, including some of the aforementioned user experience enhancements (such as Mini Map and Page Overview) will remain closed source. This will enable developers to build, use and test the browser on S60 phones, and on the Windows emulator in the S60 SDK. We also intend to change our operational model so that Nokia developers can directly work together with the Open Source community.
We hope that the combined development effort of Nokia, Apple, and the Open Source community on the browser engine will quickly adopt new Web technologies and improve compliance with future Web standards. This will create a low cost, high quality and cross-platform Open Source browser engine for all. Furthermore other mobile software and device vendors will be able to use the mobile browser engine for their platforms thus reducing the browser fragmentation on mobile devices. It is our hope that the mobile WebKit browser engine becomes the de facto Open Source engine for mobile browsers to drive innovation in content, services, browser features and technology in the mobile industry.
Conclusion
We developed a Web browser for mobile devices for the S60 software platform that is based on Open Source components. Our architecture is based on the Open Source browser engines WebCore and JavaScriptCore from Apple that were originally developed by the KDE community. To improve usability we developed a novel method called Mini Map that allows users to view full Web pages on their mobile devices in a similar way like on their desktop.
We believe that our work redefines the state-of-the-art for mobile Web browsers in terms of usability as well as technical realization. We invite the Open Source and research community to work together with Nokia and others to improve browsing for future mobile devices making new features available to millions of people – in the future possible more than desktop and laptop computers combined.
The Nokia Open Source Browser will ship on S60 3rd edition devices, and some will be available already in the first half of 2006. The described browser is already available as part of the S60 3rd Edition SDK, available for download from Forum Nokia.
Acknowledgments
Our thanks to Antti Koivisto and Virpi Roto, who are among the main authors of the Mini Map method, and to Yael Aharon and Kimmo Kinnunen for their insightful reviews of this paper.
References
[App06] Apple, Safari Web Browser, http://www.apple.com/macosx/features/safari/, 2006
[Bau04] Baudisch P., Xie X., Wang C., Ma W. Collapse-to-Zoom: Viewing Web Pages on Small Screen Devices by Interactively Removing Irrelevant Content. Proc. ACM UIST 2004.
[Buy02] Buyukkokten, O., Kaljuvee, O., Garcia-Molina, H., Paepcke, A., Winograd, T. Efficient Web Browsing on Handheld Devices Using Page and Form Summarization. ACM Transactions on Information Systems, Vol. 20, No. 1, January 2002, p. 82–115.
[Gup03] Gupta, S., Kaiser G., Neistadt D., Grimm, P. DOM-based content extraction of HTML documents. WWW 2003: 207 –214.
[Hoi03] Hoi K.K., Lee D.L., and Xu J. Document Visualization on Small Displays. Proc. Int’l Conf. Mobile Data Management, ACM Press, 2003: 262–278.
[Fel05] Feller, J. What Users Really Do With Their Mobile Phones? Nokia Study, http://www.netlab.tkk.fi/opetus/s383001/2005/kalvot/6.shtml, 2005
[Ful01] Fulk, M. Improving Web Browsing on Handheld Devices. Proc. ACM CHI 2001.
[Kde06] KDE Project, Konqueror Web Browser, http://www.konqueror.org/, 2006
[Lam05] Lam H., Baudisch P. Summary Thumbnails: Readable Overviews for Small Screen Web Browsers. Proc. ACM CHI 2005, p. 681-690.
[Lie05] Lie, Håkon. Browsing on small (and not-so-small) screens. Proc. Of XTech 2005: XML, the Web and beyond.
[Mil02] Milic-Frayling, N., Sommerer, R. SmartView: Flexible Viewing of Web Page Contents. Poster paper at WWW 2002, http://www2002.org/CDROM/poster/172/
[Oma01] Open Mobile Alliance, XHTML Mobile Profile, http://www.openmobilealliance.org/tech/affiliates/wap/wap-277-xhtmlmp-20011029-a.pdf, 2001
[Rot03] Roto, V., Kaikkonen, A. Perception of Narrow Web Pages on a Mobile Phone. Proc. Human Factors in Telecommunications 2003.
[Rot06] Roto, V., Popescu, A., Koivisto, A., Vartiainen, E.: Minimap – a Web Page Visualization Method for Mobile Phones. Proc. ACM CHI 2006.
[S60] S60, Web Browser for S60, http://s60.com/browser, 2006
[Wap98] Wireless Application Protocol Forum, Wireless Application Protocol Wireless Markup Language Specification, http://www.wapforum.org/what/technical/wml-30-apr-98.pdf, 1998
[Wob02] Wobbrock J., Forlizzi J., Hudson S., Myers B. WebThumb: Interaction Techniques for Small-Screen Browsers. Proc. ACM UIST 2002.
[Yan03] Yang C. and Wang F. Fractal summarization for mobile devices to access large documents on the web. WWW 2003: 215-224.