A. Graham, H. Garcia-Molina, A. Paepcke, and T. Winograd. Time as essence for photo browsing through personal digital libraries. In JCDL ’02: Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, pages 326–335, New York, NY, USA, 2002. ACM. [PDF]

———-

This paper decribes PhotoBrowser, a prototype system for personal digital pictures that organize the content using the timestamps of each photo. The authors’ main assumption is that text annotations are great because are accessible to a variety of search and processing algorithms. Many systems exploits this possibility requiring time-intensive manual annotations (e.g., FotoFile [Kang and Shneiderman, 2000] and PhotoFinder Kuchinsky et al., 1999]).

The authors’ proposal is that of exploiting the timestamps at which the pictures have been taken. They propose an organizational, and visual, clustering of the pictures which divide the photos using longer periods of inactivities between one picture and the next.

To verify their assumption, the authors tested the PhotoBrowser against other two applications. The first was a Hierarchical Browser, which organized the pictures at different levels corresponding to time at different granularity. The Scrollable Browser, organized the pictures into a single time-ordered list that could be scrolled.

The author asked 12 subjects to perform 6 retrieval tasks following different criteria. Their main result was that the browser that organized the pictures into temporal clusters enabled significantly faster completion times at an average of about 50 seconds.

These last months, I have been collaborating to a research project on Multimodal Information Retrieval of digital pictures collected through camera phones. Recently, one of the papers resuming the results of the research was presented at the International Workshop of Mobile Information Retrieval, held in conjunction with SIGIR in Singapore. Here goes the abstract and the URL to download the paper.

X. Anguera, N. Oliver, and M. Cherubini. Multimodal and mobile personal image retrieval: A user study. In K. L. Chan, editor, Prooceeding of the International Workshop on Mobile Information Retrieval (MobIR’08), pages 17–23, Singapore, 20-24 July 2008. [PDF]

Mobile phones have become multimedia devices. Therefore it is not uncommon to observe users capturing photos and videos on their mobile phones. As the amount of digital multimedia content expands, it becomes increasingly difficult to find specific images in the device. In this paper, we present our experience with MAMI, a mobile phone prototype that allows users to annotate and search for digital photos on their camera phone via speech input. MAMI is implemented as a mobile application that runs in real-time on the phone. Users can add speech annotations at the time of capturing photos or at a later time. Additional metadata is also stored with the photos, such as location, user identification, date and time of capture and image-based features. Users can search for photos in their personal repository by means of speech without the need of connectivity to a server. In this paper, we focus on our findings from a user study aimed at comparing the efficacy of the search and the ease-of-use and desirability of the MAMI prototype when compared to the standard image browser available on mobile phones today.

T. Tan, J. Chen, P. Mulhem, and M. Kankanhalli. Smartalbum: a multi-modal photo annotation system. In MULTIMEDIA ’02: Proceedings of the tenth ACM international conference on Multimedia, pages 87–88, New York, NY, USA, 2002. ACM. [PDF]

——-

Applications supporting annotation of pictures with voice: SmartAlbum (Tan et al., 2002) that unifies two indexing approaches, namely content-based and speech, Chen et al. (2001) proposed the use of a structural speech syntax to annotate photographs in four different fields, namely event, location, people, and date/time. Show&Tell (Srihari et al., 1999), which uses speech annotations to index and retrieve both personal and medical images, and FotoFile (Kuchinsky et al., 1999), which extends annotation to a more general multimedia object.

This demonstration presents a novel application (called SmartAlbum) for photo indexing and retrieval that unifies two different image indexing approaches. The system uses two modalities to extract information about a digital photograph; i.e. content-based and speech annotation for image description. The result is a powerful image retrieval tool that has capabilities beyond what current single-mode retrieval systems can offer. We show on a corpus of 1200 images the interest of our approach.

A. Girgensohn, J. Adcock, M. Cooper, J. Foote, and L. Wilcox. Simplifying the management of large photo collections. In Proceedings of INTERACT’03, pages 196–203. IOS Press, 2003. [PDF]

——–

This paper contains useful references on the fact that event segmentation of pictures was observed to be a valid criterion for organizing pictures The paper reports an algorithm that was compared with those of Graham at al. (2002), Platt et al. (2002), and Loui and Savakis (2000).

With digital still cameras, users can easily collect thousands of photos. Our goal is to make organizing and browsing photos simple and quick, while retaining scalability to large collections. To that end, we created a photo management application concentrating on areas that improve the overall experience without neglecting the mundane components of such an application. Our application automatically divides photos into meaningful events such as birthdays or trips. Several user interaction mechanisms enhance the user experience when organizing photos. Our application combines a light table for showing thumbnails of the entire photo collection with a tree view that supports navigating, sorting, and filtering photos by categories such as dates, events, people, and locations. A calendar view visualizes photos over time and allows for the quick assignment of dates to scanned photos. We fine-tuned our application by using it with large personal photo collections provided by several users.

R. Wasinger and A. Krüger. Modality preference – learning from users. In Proceedings of the User Experience Design for Pervasive Computing workshop, Munich, Germany, 11 May 2005. [PDF]

——-

This paper describes a qualitative comparison of input modalities for mobile applications. The author conducted a user study with about 50 users that were asked to fill questionnaires on a PDA. The questions targeted products that the users could find in a shop and therefore had to be answered while on the place. The users were divided in different groups and each group was exposed to different input modality: speech, handwriting, intra- and extra-gestures.

Interestingly, speech was reported as being the most comfortable modality to use in comparison with handwriting but also the modality that could mostly expose privacy. Users reported being uncomfortable speaking sensible information to their devices. Handwriting was see as conflicting with other manual activities that needed to be carried out while shopping.

J. Kustaniwitz and B. Shneiderman. Motivating annotation for personal digital photo libraries: Lowering barriers while raising incentives. Technical Report HCIL-2004-18, University of Mariland, College Park, MD, USA, 2005. [PDF]

——–

The main argument of this paper is that annotation is a tedious activity. Few users take the time to annotate carefully and sistematically their collection and therefore thousands of pictures lie unorganized in folders pretty much as printed pictures lie in shoeboxes. The author highlights how part of the problem is that users themselves do not understand what they can do differently once pictures are annotated.

Toward the end of raising motivation for tagging pictures, this paper describes several applications for visualizing collections of annotated pictures. For instance, Birthday Collage is an application that automatically pulls together pictures taken at birthdays for the same person.

The value of personal digital photo libraries grows immensely when users invest effort to annotate their photos. Frameworks for understanding annotation requirements could guide improved strategies that would motivate more users to invest the necessary effort. We propose one framework for annotation techniques along with the strengths and weaknesses of each one, and a second framework for target user groups and their motivations. Several applications are described that provide useful and information-rich representations, but which require good annotations, in the hope of providing incentives for high quality annotation. We describe how annotations make possible four novel presentations of personal photo collections: (1) Birthday Collage to show growth of a child over several years, (2) FamiliarFace to show family trees of photos, (3) Kaleidoscope to show photos of related people in an appealing tableau, and (4) TripPics to show photos from a sequential story such as a vacation trip.

B. Schneiderman and H. Kang. Direct annotation: A drag-and-drop strategy for labeling photos. In IV ’00: Proceedings of the International Conference on Information Visualisation, pages 88–96, Washington, DC, USA, 2000. IEEE Computer Society. [PDF]

———–

The basic argument of the authors of this paper is that large pictures collections need annotation in order to support efficient retrieval. However, annotation is a tedious and time-consuming task. The authors present in this paper a basic HCI technique which allows the user to annotate without much typing. Keywords can be inputed once-for-all through a list and becomes available to be dragged and dropped over pictures.

The system has the advantage of supporting labeling of regions of the pictures (see figure below) but has the disadvantage of reduced scalability.

Annotating photos is such a time-consuming, tedious and error-prone data entry task that it discourages most owners of personal photo libraries. By allowing users to drag labels such as personal names from a scrolling list and drop them on a photo, we believe we can make the task faster, easier and more appealing. Since the names are entered in a database, searching for all photos of a friend or family member is dramatically simplified. We describe the user interface design and the database schema to support direct annotation, as implemented in our PhotoFinder prototype.

Biologists have constructed a genetic map of Europe showing the degree of relatedness between its various populations. This genetic map contains many similarities to the geographical map. The major genetic differences are between populations of the north and south Europe.

Researchers say that the patters of genetic differences reflects the colonization history of Europe, which dates back 45,000 years ago. The map also identify the existence of two genetic isolated groups: the Finns and the Italians.

The former arose because the Finnish population was at one time very small and then expanded, bearing the atypical genetics of its few founders. The latter is between Italians (yellow, bottom center) and the rest. This may reflect the role of the Alps in impeding free flow of people between Italy and the rest of Europe.

[more]

This week the long-awaited Google Phone produced by HTC was approved by the American FCC commission for use in the United States. This means that commercialization might start as early as October this year.

I thought that was a good idea to write a short post about it because we reached a critical mass of demos and applications circulating the Net. Google philosophy is clear and it is opposite of that of Apple or Nokia: using open-sources solutions and leveraging on open source movements to create the critical ecology around their product.

They also pulled together a nice team of designers, who released a clever sets of features for the interface. This phone will integrate nicely Google’s product that we use and love, like gMaps, gMail and so on. However it will also provide new experience opportunities for the users. For instance, the HTC device should incorporate a compass that should allow an augmented reality experience of gMaps’ Street View. Turning around the device should adapt the view of the map so to that the virtual experience will closely resemble and adapt the virtual experience. Plus the application will overlay the names of the streets to the view, giving useful information to the user.

The gPhone is also full of clever UI features that are going to make the user experience both richer and simpler. For instance, access to notifications was aggregated in a single part of the interface: a pull-down tile on the menu bar.

In sum, I am looking forward to put my hands on the first commercial model and make a full review of the system. For now, I celebrate to what resembles to be clever design decisions.

One reason Helena and I would never be close friends is that I am about half tall as she. People tend to stick to their own size group because it’s easier on the neck. Unless they are romantically involved, in which case the size difference is sexy. It means: I am willing to go the distance for you.

Miranda July. No one belongs here more than you. Canongate, Edinburgh, UK, 1997.

[web site]