I found this interesting information on Beppe Grillo’s blog. It is an official document by Ryanair summarizing the revenue margins for different companies where Alitalia is taken as the lowest point in the scale 🙁
Here is clearly visible how the company is loosing money every single day. In the mean time the direction of the company keep having the highest salary in europe. Povera Italia!
I have been quite this period trying to figure out a way to group visually messages that refers to the same keyword. After a couple of bad attempts I tried to visualize a polygon connecting the messages referring to the same keyword. Subsequently, I tried to find the centroids of the polygon using this nice formula. Then I decided to attach the keyword to this “center of gravity”. This seems to be simple but it was damn hard, as I had to find the good sequence to connect the points of the polygon to avoid self interceptions of the lines.
Anyway, there are still some flaws in the code but here is the first rendering to comment and destroy. All comments are welcome.
Tags: clustering, information metric, information retrieval, information visualization, Location Based Services, map algorithms, maps, spatial clustering, text data mining
StickyShadows is another Location Based System that allows the user to leave a virtual message to a certain physical place. Their base concept is to give more relevance to alerts to prompt the retrieval of the information.
Tags: Location Based Services
James Clar has realised a three-dimensional display cube that permits to visualize three dimensional interactions or objects moving in space. I have always been fascinated by simple forms and this one it is. Maybe this simple idea can help solve some of the issues of three dimensional projectors and holography.
Tags: hack, information visualization
The contextual network graph does not encode any information about grammatical or hierarchical relationships between terms. Its structure is determined purely by term co-occurrence across the collection.
Each edge in the graph has a strength assigned to it whose magnitude depends on our choice of the local and global term weighting scheme used in generating the TDM. The only constraint on weighting schemes is that all edge weights must fall in the interval (0,1).
From:
M. Ceglowski, A. Coburn, and J. Cuadrado. Semantic search of unstructured data using contextual network graphs. Preliminary white paper, National Institute for Technology and Liberal Education, Middlebury College, Middlebury, Vermont, 05753 USA, 2003.
Tags: Latent Semantic Analysis
M. Ceglowski, A. Coburn, and J. Cuadrado. Semantic search of unstructured data using contextual network graphs. Preliminary white paper, National Institute for Technology and Liberal Education, Middlebury College, Middlebury, Vermont, 05753 USA, 2003. [url]
————————————
This paper describe the Contextual Network Graph, a technique that should solve some of the pitfalls of the latent semantic indexing, like the poor scalability of the singular value decomposition algorithm.
The authors offer an alternative interpretation of the term-document matrix (TDM), which is essentially a lookup table of term frequency data for the entire document collection. In LSI this is interpreted as a high-dimensional vector space. Alternatively, this can be seen as a bipartite graph of term and document nodes where each non-zero value in the TDM corresponds to an edge connecting a term node to a document node. In this way, every term is connected to all of the documents in which the term appears, and every document has a link to each term contained in the document. The authors call this a contextual network graph.
This construct correspond to the intuition that documents that share many rare tems are likely to be semantically related.
Their idea is to use this representation to search a document collection energizing a query node and allowing the energy to propagate to other nodes along the edges of the graph based on a set of simple rules.
Their results show comparable results to an LSI search engine. In a 1981 dissertation at the University of Illinois, Scott Preece describes an almost identical technique under the name spreading activation search.
Tags: Exploratory Data Analysis, information retrieval, Latent Semantic Analysis, text data mining
At this stage I feel pushed more by exploration and curiosity than a precise strategy. I know where I have to go and which point I have to reach, but on the exact path to take things are still uncertain. Now I can see a couple of options:
step zero -> Computing the semantic distance matrix using Latent Semantic Indexing.
1. Using this matrix to sparsify the gabriel graph of the geographic layer;
2. Merging the semantic and the geometrical matrix to be used in an ad-hoc clustering algorithm;
3. Computing the semantic or geometric clustering and apply the other matrix to mining the results.
Tags: clustering
Finally I managed to compute the b-skeleton of the messages left of the campus of EPFL. The idea is to use this visualization to find the messages that are more close to the others. Additionally this graph can be sparsified, meaning that some long / irrelevant connections can be removed, clustering the zones in sub-graphs. This technique is also useful to find outliers.
Beta skeletons: A continuous family of proximity graphs proposed by Kirkpatrick and Radke [*]. These graphs are defined by a positive real parameter Beta. Lower values of Beta give denser graphs (with more edges). There are two types of Beta-skeletons: lune-based (the neighborhood is an intersection of discs), and circle-based (the neighborhood is a union of discs). For Beta=1 the skeleton is in fact the Gabriel graph, and for Beta=2 the (lune-based) skeleton is the Relative Neighborhood graph.
[*] DB Kirkpatrick and JD Radke.
A framework for computational morphology.
In GT Touissant, editor, Computational Geometry, pages 217-248. Elsevier, 1985.
Spent a considerable amount of time today to manipulate the campus map to display the position of the messages. I have partly implemented a class that verify the connections of the messages based on a given criteria. Tomorrow will be the right time to complete this initial part. Good night.
