Originally from here. I found this illuminating and hilarious for my research on search engines.

Imagine that you are curious about what people typically order for breakfast down at your local diner, and you want to display this information in visual form. You decide to examine all the breakfast orders from a busy weekend day, and record how many times the words bacon, eggs and coffee occur in each order.

You can graph the results of your survey by setting up a chart with three orthogonal axes – one for each keyword. The choice of direction is arbitrary – perhaps a bacon axis in the x direction, an eggs axis in the y direction, and the all-important coffee axis in the z direction. To plot a particular breakfast order, you count the occurence of each keyword, and then take the appropriate number of steps along the axis for that word. When you are finished, you get a cloud of points in three-dimensional space, representing all of that day’s breakfast orders.

Copyright notice: the present content was taken from the following URL, the copyrights are reserved by the respective author/s.

Tags: Latent Semantic Analysis, text data mining

Digging in my memories, I have remembered Nathan Never, a comic hero that used to fill some blanks of my teenage. I have always been a fan of SF stuff and NN was a good one.

Mathemagenic has a good review and summary of a bounch of papers presented during the AOIR conference that target learning and blogging.

My selection of the selection:

– Trena Paulus & Vanessa Dennen, Weblogs as spontaneous mentoring mechanism for academics – very good overview of “Profworld” academic blogging community (qualitative, with examples) and discussion on how mentoring takes place there (really hope that paper or slides will be online)

– Jang Hyun Kim, Blog as an oppositional medium?: A Semantic Network Analysis on the Iraq War Blogs

– Ericka Menchen Trevino, Blogger motivations: Power, pull, and positive feedback (full text is here) – a study on blogging motivations between college students. I was especially fascinated by the framing the results on negotiating anonymity and Ericka was very nice to blog it

– Trena Paulus, Vanessa Dennen, The next generation of research methods for online collaborative learning environments

Tags: learning technology, pedagogy, blogs

This idea follows some research into memory that says that the probability of an event to happen is inversely proportional to the time that passed since it last happened.

The idea started from the assumption that memory contains concepts. These are associated. Those that are more ‘activated’ are easier to recall. The activation of a given concept depends on two things: the base activation of that concept and the sum of the activation of associated concepts multiplied by the strength of association.

Copyright notice: the present content was taken from the following URL, the copyrights are reserved by the respective author/s.

Tags: information retrieval, information visualization, text data mining

M. F. Caropreso, S. Matwin, and F. Sebastiani. Text Databases and Document Management: Theory and Practice, chapter A learner-indipendent evaluation of the usefulness of statistical phrases for automated text categorization, pages 78–102. Idea Group Publishing, Hershey, US, 2001. [url]

——————————

This paper present a technique and proof to use a n-grams indexing technique for Text Classification (TC). A phrase is a textual unit usually larger than a word but smaller than a fuill sentence. Phrases have a smaller degree of ambiguity than their constituents words, thanks to the mutual disambiguation effect of words.

An n-gram is an alphabetically ordered sequence of unigrams. The authors’ learner-indipendent study has shown that feature evaluation functions routinely used in the text categorization experiments tend to score many bigrams higher than unigrams that they would themeselves select in unigram-only feature selection task, sometimes giving rise to a too high bigram “penetration level”.

Tags: machine learning, text data mining

KidPad is a collaborative story authoring tool for children. It provides basic drawing functionality on a zooming canvas enabled by Jazz. The narrative structure of a story is defined by creating spatial hyperlinks between objects on the canvas. Instead of using a standard WIMP (Windows, Icons, Menus, Pointer) user interface, KidPad uses local tools that can be picked up, used and dropped anywhere on the drawing surface. The local tools interface and MID, a Java library developed at the University of Maryland, allows KidPad to support shoulder-to-shoulder collaboration. If multiple USB mice are connected to the computer each mouse will control a tool in KidPad, making it possible to let several children simultaneously create a story together!

Tags: human computer interaction, interaction design

I found an interesting introductory course on this theme (via), which summarize different techniques for decision rule class membership attribution. These can be divided into parametric and non parametric depending on the fact that the choice for the assignment is computed using the probability or an euclidean metrics criteria.

Interestingly enough, besides being extremely simple to implement, the probability of error of those non-parametric variants, is sufficiently close to the Bayes (optimal) probability of error. Sergei Savchenko, presents in the document cited above, a compilation of those methods, in an order based on the efficiency and the accuracy for the information retrieval.

The first presented is the k-nearest neighbor decision rule, which classify an unknown object to the class most heavily presented in its nearest neighbors.

The Voronoi diagram is a partition of space into regions, within which all points are closer to some particular node than to any other node.

The Dalaunay triangulation is a dual of the Voronoi diagram. It two Voronoi regions share a boundary the nodes of those regions are connected with an edge. These nodes are called the Delaunay neighbors.

The Gabriel graph of a series of points is a subgraph of Delaunay triangulation for that set. Gabriel editing reduces the size of the training set even more than Voronoi editing.

Tags: information retrieval, information visualization

S. Deerwester, S. T. Doumas, T. K. Landauer, G. W. Furnas, and R. Harshman. Indexing by latent semantic analysis. Journal of the Society for Information Science, 41(6):391–407, 1990. [url]

—————————-

This is the first technical LSI paper. It offers a good background on the previous theory and on the technical choices behind the algorithm.

Essentially, the authors developed the technique to overcome the deficiency of current information retrieval methods, that is the words searchers often use are not the same as those by which the information they seek has been indexed. The two side of the issue they describe are the synonymy and the polysemy. The former is the fact that there are many ways to refer to the same concept. The latter is the fact that most words have more than one distinct meaning.

The criteria they used to distinguish between different models are: adjustable representational richness (hierarchical clustering is too restrictive; they looked for models whose power could be varied); explicit representation of both terms and documents (as in two-mode factor analysis or tree unfolding); computational tractability for large datasets.

The latent semantic structure analysis starts with a matrix of terms by documents. Then this is analyzed by Singular Value Decomposition (SVD), to derive the particular latent semantic structure model. The process divide the original matrix into three representative matrix, which contains “eigenvectors” and “eigenvalues”, and which show a breakdown of the original data into linearly indipendent components or factors.

Three particular comparisons were of interest for the authors: the comparisons of two terms; two comparisons of two documents; and a comparison of a term and a document.

Tags: information retrieval, Latent Semantic Analysis, Singular Value Decomposition

I tried an egocentric search on this web service that promise to take the best of the search engines. One of their claims is that only 1,1 % of the top links are shared between the four search engines (MSN, Google, Yahoo, AskJeeves).

Tags: clustering, google, information retrieval, information visualization

Today I had a meeting with my advisor on possible direction to move the project forward. We discussed the option to use a semantic distance between the messages as another dimension for the triggering of significant connections among the messages. One of the options would be to parametrize the influence of the semantic or the geographical distance on the final clustering of the information. The distortion over one of these two dimensions should be finally decided by a combination of automatically computed factors and the user feedback.

Two ways are possible to incorporate the user feedback. The first is asking for explicit feedback (i.e., Is it what you were looking for – YES —- NO); the second way is to use implicit feedback measurements to asses whether the behavior of the user is affected by the visualization and how this respond to the goodness of the visualization itself.

Tags: human computer interaction