With MonitorThis you can subscribe to 22 different search engine feeds at the same time. Enter a search term and click the ‘make monitor.opml’ button to get a list of rss feeds in OPML format.

Tags: blogs

Mobup is a small J2ME application that manages photo uploads on Flickr from your mobile device built using the Flickr APIs. Once installed it gives you the possibility to shoot your photo and add title, tags and description from the same applications with optimal user experience and to post the shooted photo on your blog.

Tags: blogs, graffiti, hack, Location Based Services

D. Kelleher. Spam filtering using contextual network graphs. Master’s thesis, Computer Science Department, Trinity College, Dublin, Ireland, 2004. [pdf]

——————-

This paper present a comparaison between three differnet classification methods based on Naive-bayesian, Latent Semantic Indexing and Contextual Network Graph for the purpose of improving common spam filtering mechanisms.

The author show how LSI and CNG outperform the Naive-Bayes Classifier for the ability that those two methods have of dealing with text polysemy and synonymy. Polysemy reduces in fact the strenght of keywords by increasing the number of classes in which a polysemous word can appear. Synonymy reduces the strenght of keywords by spreading their value among a number of synonyms. The two methods above have a powerful approach to classification of natural language text because they use a context-based search that take into account the semantic links between words and search over a semantic space rather than simply a list of keywords.

The foundamental problem of the naive-bayes probabilistic approach is that it makes the assumption that each word in a document is indipendent from the others. This independence assumption makes this classifier unsuitable for natural language processing since it loses the contextual relations between words and fails to address the issues above.

The author uses its own implementation of the above methods using a public available dataset and a Stop-Word removal. The implementation was done in JAMA a Java Matrix Package.

The success of a certain method was measured using some machine learning markers: a k-fold cross validation testing mechanism.The author report how the most important results from the test was not the accuracy, but the fact that the high complexity of SVD makes its operation significantly slower compared to the other two.

Additionally, results showed than the ratio between the initial energy activation and the energy threshold of CNG had to be adjusted to maintain the accuracy while increasing the corpus size. This may be related to the fact that growing the dataset the threshold value should be adapted as suggested also by Kimoto and Iwadera [1989](p.236): ‘this suggest that the threshold value should increase with trasversal distance.’

In conclusion the results shows that CGN should be prefered over LSI because of the computational speed and because it is not patented. 

Tags: information retrieval, Latent Semantic Analysis, Singular Value Decomposition, spreading activation, statistics, text data mining

Waiting for the Ajax-Web2.0 revolution, I found this intermediary solution that allows to bring around great collection of softwares without the weight of a laptop or an handheld. Sometimes, in fact, I found myself in the situation of needing a decent piece of software on a machine I was working on without the hassle to have to install the thing.

Recently I started noticing a growing number of projects to port common software apps into a USB stick format so that can be run cross platform. Here is a collection I assembled recently:

• Chat/instant messaging: Miranda, Trillian Anywhere
• Browsers: Portable Firefox
• Password storing: Keepass, Pass2Go
• Operating Systems: Linux, Windows CE, VPM (Linux)
• System tools: BBLean,
• PIMs/organisers: EssentialPIM, Sunbird
• Email: JBMail, Portable Thunderbird, EmailVoyager, PocoMail PE
• Encryption: CryptoAnywhere, PuTTY
• Office: AbiWord, OpenOffice (instructions in a PDF here), Papyrus
• Sticky Notes: ATNotes
• Music: Zoom Player, XMPlay, Coldplayer, Foobar2000
• Web Authoring: NVU
• Programming: Movable Python
• Web services: Miniserver (Apache, PHP server)
• Email: iScribe, Foxmail
• Task management: ToDoList
• Wiki: Pimki, TiddlyWiki
• Games: SteelSK, Fotaq
• PDF Reader: RD

Tags: hack

H. Kimoto and T. Iwadera. Construction of a dynamic thesaurus and its use for associated information retrieval. In Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, pages 227 – 240, Brussels, Belgium, 1989. ACM Press, New York, NY, USA. [pdf]

——————–

This paper describe the AIRS system (Associated Information Retrieval System), where for the first time (?) a dynamic thesaurus was used in a associative retrieval system. Basically the authors built a term by term matrix of keywords that was assemble by putting together information from a user’s sample of relevant documents and a static thesaurus.

The dynamic thesaurus was constructed on a network structure where each node was representing a keyword, while each link was representing a relationship between keywords. A weight was assigned to each node and to each link between the nodes. Four different thesaurus were therefore derived combining the presence of the keywords weighting of nodes and links. This served the experimental evaluation.

Results suggested that node weighting was effective for improving the Keywords precision. Additionally, the finding suggested that the activation threshold value of the node should increase with the transversal distance of the network routing. Finally the use of the thesaurus with weighted nodes and links showed an increase of precision rate and recall rate.

Tags: information retrieval, spreading activation

There are a couple of scenarios that I have been developing to increase the usage of ShoutSpace at EPFL. I am in the need of Data so, every application that could bring me messages is appealing to me.

Scenario 1: Maintenance Group at EPFL. The idea is to give SS to the maintenance group so that they can use it as a support to coordinate their intervention on the site. Ideal would be that they could use it consistently over a certain period of time to optimize the working schedule. A Typical usage can include the items to be repaired and those that have been repaired already. Also they could annotate rooms that need to be refurnished. Exhibition panels that need to be moved and where. Here the mix of annotation and place and coordination takes its full power.

Scenario 2: Information Kiosk at EPFL. The idea is to translate SS into a Java Applet that can be executed from the information points at EPFL. There people could log-into the system and browse the campus looking for specific localized informations. For instance, where is a cafeteria near by. Additionally they can leave notes to coordinate for meetings on campus. People uses the kiosks to look for phone numbers or information about courses. In the same way they can use SS to look for specific information about features of places. Or to read the annotations left by other users.

Scenario 3: Newsgroup of EPFL into ShoutSpace. The idea is to import the EPFL newsgroup into ShoutSpace so that they can be mapped to the point of emittance or reception. Some specific newsgroup like Petit-Announces are extremely geographical in which they target specific points of interests (i.e., the room of the poster in case of selling/buing message or the zone interested by a particular maintenance). Offering these messages with a map support seems to be a great add-on. Additionally it would increase the value of using SS in daily routines.

Scenario 4: ShoutSpace as a support for event organization. Periodically events are organized at EPFL for the students life (i.e.,  le Balelec). A great deal of spatial management is needed to coordinate these efforts of moving materials, coordinating security, etc. The ideal would be for the organizers to use SS as a support tool for the organization of the event.

Scenario 5: ShoutSpace as support for an urban planning class of ENAC. The architecture faculty of EPFL has courses of Urban Planning. One possible class work could be the study of the EPFL site as a micro-city environment, defining the critical aspect of the site that could be redesigned. The class could use SS for the semester, leaving messages and annotations connected with their observation period. Subsequently the notes could be reported in the classroom for teaching and discussion purposes.

 

1- What is the current research on SA? Is it used? Why it did not have the some success of others techniques?

2- What are the advantages of using SA over SVD (Scalar Vector Decomposition)?

3- At the methodological level how is it possible to evaluate the performance of a IR system over another, given the fact that some of the parameters used for these measures are human based (i.e., defining what is relevant in a given collection returned by a query)?

Please feel free to chime in!

Tags: information retrieval, spreading activation

Recall is the proportion of all documents in the collection that are relevant to a query and that are actually retrieved.

Precision is the proportion of the retrieved set of documents that is relevant to the query.

The fallout rate is the ratio between number of non relevant documents retrieved and the total number of non relevant documents in the collection.

Tags: information retrieval

F. Crestani. Application of spreading activation techniques in information retrieval. Artificial Intelligence Review, 11(6):453–482, December 1997. [pdf]

————–

This paper contains a literature review on the theme of spreading activation techniques used in information retrieval, which is a prticular instance of associative retrieval. The paper offers, at first, an overview of previeus research on this theme focussing on the associative retrieval with spreading activation.

One of the most common technique to build the associative network used by this technique is to expanding the original query with term-term, term-document and document-document associations. This technique is based on the assumption that there are statistically determinable relations among the tems, and between the terms and the documents. This assumption is also accounted for some of the drawback of the methods, because it is questioned the general applicability of the similarity measures derived statistically in domains which are extern to the knowledge domain. Additionally these similarities are computed under the assumption that the terms in the documents are originally uncorrelated.

The rest of the paper shows a couple of applications using Spreading Activation for IR. Some of the result shows a lack of consisten improvement of the effectivenes of the retrieval over other methods (i.e., vector processing model). Additionally, one of the drawback of SA is the difficulty of building and maintaining the network of connections between the elements (i.e., documents and terms).

One interesting application is that of Kimoto and Iwadera, AIRS, which propose to use an SA technique in connection with a dynamic thesaurus, which is a network where nodes represent terms and links represent semantic relationship between terms.

Another interesting idea proposed in the paper is that of using automatic constructed hypertext to facilitate the IR with the SA technique. 

Tags: information retrieval, Latent Semantic Analysis, machine learning, Singular Value Decomposition, text data mining

I found these three degree of good formatting for visual presentation:

[1] Legibility – distinctness that makes perception easy

[2] Readability – writing (print or handwriting) that can be easily read

[3] Clarity – free from obscurity and easy to understand; the comprehensibility of clear expression

Therefore the first level belong to the perception, the send to the presentation and the third to the content.