Yoono is a free software application you can download which combines for the first time the management and sharing of information. Based on pooling user knowledge, yoono is a collaborative search engine and an innovative communication tool.

Yoono’s search results are obtained from the bookmarks of web users. Searces are independent of the user’s language sine yoono does not index keywords on the web but the bookmarks of the users of yoono.

Search results are ordered by popularity. The popularity or audience of a web site is the number of times it has been added to the yoono community for this search. Yoono also offers you a list of experts on the subject of the search. It identifies an expert using the your search – if an expert has published the url in a folder, he is identified in the results by nickname. The experts are ordered by level of expertise which corresponds to the number of users subscribed to the published folder.

When you download yoono, you also participate in yoono’s search results

Tags: collaboration tools, folksonomy, google, information retrieval

MozDex is a search engine seeded from the dmoz.org directory. MozDex uses open source search technologies to create an open and fair index. Their goal is to index the entire web in html content. They want to be able to provide a powerful and open search service to the community.

Some people may say that providing the insight into the results will offer cheaters a better way to get higher ranks, but our view is it allows us to openly discover and communicate new methods and algorithms that give a better view and representation that is less Fallible to cheaters.

Proprietary systems have already been done. We are here to utilize open technologies and open source to build an index that doesn’t use proprietary software, processes or algorithms. Freedom of information and how that information gets to you is what mozdex is about. Mozdex is built around the Nutch Search technology. Thanks to the developers and companies listed in Nutch Credits for making this software and project possible.

Tags: google, information retrieval, politics, open source

With the increasing number of tools for Information Retrieval, it is very important to find common tools for analyzing the ‘goodness’ of the performances with the end of making comparisons. When searching and retrieving documents there are four groups of results that are of interest for quantitative analysis:

• Relevant documents that were retrieved by the system (we call group A)
• Irrelevant documents that were retrieved by the system (we call group B)
• Relevant documents that the system missed (we call group C)
• Irrelevant documents that were not retrieved by the system (we call group D)

	Relevant	Irrelevant
Retrieved	A	B
Not retrieved	C	D

Intuitively a good system should try to maximize the retrieval of relevant documents, and minimize the retrieval of irrelevant documents. Recall and Precision are the fundamental parameters defining the behavior of an information retrieval system and are defined as follows:

Recall: defines the number of relevant documents retrieved as a fraction of all relevant documents

Precision: defines the number of relevant documents as a fraction of all documents retrieved by the system. Precision defines the level of noise in the information presented to the user.

Research has shown that is very difficult to achieve a high level of recall without sacrificing precision (Y. Yang et al). At the recall rate increases, the precision deteriorates very rapidly. This trend is not linear depending on the type and quality of the information and the retrieval algorithm precision. Additional metric focus on what the retrieval engine has missed instead of what is retrieved.

Missed documents: defines the number of relevant documents missed by the search engine. It is the inverse of the recall rate.

False Alarm: the measure of the level of noise in the output.

Because of the inverse relation between recall and precision, some suggest calculating the harmonic mean of the two. This is also called F-measure in the literature.

The above formula is also known as the F1 measure, because precision and recall are evenly weighted. This belong to a general formula that is:

Two other commonly used F measures are the F0.5 measure, which weights precision twice as much as recall, and F2 measure, which weights recall twice as much as precision.

Additionally, the Mean Average Precision (MAP): over a set of queries, the average precision is the precision after each relevant document is retrieved.

Where r is the rank, N the number of retrieved documents, rel() is a binary function on the relevance of a given rank, and P() is the precision at a given cut-off rank.

Tags: information retrieval, statistics

M. Sanderson and J. Zobel. Information retrieval system evaluation: Effort, sensitivity, and reliability. In Proceedings of the Special Interest Group on Information Retrieval SIGIR’05, pages 162–169, Salvador, Brazil, August 15-19 2005. ACM. [pdf]

——————-

This paper discuss the way other researcher comments their results on the effectiveness of Information Retrieval systems. Effectiveness, in this context, is computed by measuring the ability of systems to find relevant documents. Statistical significance tests like the t-test and the Wilcoxon are commonly used. These tests are based on a series of assumptions: namely that the values being tested are distributed symmetrically and normally, and that are a random sample of the population. Additionally, the tests produce type I and type II errors. A type I is a false positive with an incidence of 1 every 20 tests at p-value of 0.05. A type II is a false negative with an unknown incidence.

The results computed by the authors show that if significance tests are omitted or if the improvements are small, results are not reliable. Some conclusions sets a teting methodology:

1- they found the t-test more reliable than the the alternatives, based on the false-positive rate of 5%;

2- another criteria found is that the Mean Average Precision (MAP) difference between the tests should be bigger or equal to 20% with a positive significance test in order to demonstrate positive results for the tested IR engines;

3- for small topic set sizes (<25) observing statistical significance does not guarantee that a result will be repeatable in other sets of topics. 

Tags: information retrieval, statistics, text data mining

TREC is a series of conference organized by various member of the US Defense Department around the concept of Information Retrieval. Various datasets are made available to the public to test and develop different search engines. One of the greatest advantages of these data is the availability of a relevance measure for a given query, that was computed by a number of experts that blinded reviewed the given query against the given results.

TREC uses the following working definition of relevance: If you were writing a report on the subject of the topic and would use the information contained in the document in the report, then the document is relevant. Only binary judgments (“relevant” or “not relevant”) are made, and a document is judged relevant if any piece of it is relevant (regardless of how small the piece is in relation to the rest of the document).

Judging is done using a pooling technique (described in the Overview papers in the TREC proceedings) on the set of documents used for the task that year. The relevance judgments are considered “complete” for that particular set of documents. By “complete” we mean that enough results have been assembled and judged to assume that most relevant documents have been found. When using these judgments to evaluate your own retrieval runs, it is very important to make sure the document collection and qrels match.

Tags: information retrieval

Algorithmic stemmers continue to have great utility in IR, despite the promise of out-performance by dictionary-based stemmers. Nevertheless, there are few algorithmic descriptions of stemmers, and even when they exist they are liable to misinterpretation. Snowball, is a language defined by Porter, in which stemmers can be exactly defined, and from which fast stemmer programs in ANSI C or Java can be generated. A range of stemmers is presented in parallel algorithmic and Snowball form, including the original Porter stemmer for English.

PyStemmer provides stemmer functionality in Python for English, German, Norwegian, Italian, Dutch, Portuguese, French, Swedish. PyStemmer is based on the Snowball stemmer.

Tags: information retrieval, natural language processing, text data mining

The Cross-Language Evaluation Forum (CLEF) supports global digital library applications by (i) developing an infrastructure for the testing, tuning and evaluation of information retrieval systems operating on European languages in both monolingual and cross-language contexts, and (ii) creating test-suites of reusable data which can be employed by system developers for benchmarking purposes.

Through the organisation of system evaluation campaigns, the aim is to create a community of researchers and developers studying the same problems and to facilitate future collaborative initiatives between groups with similar interests. CLEF will also establish strong links, exchanging ideas and sharing results, with similar cross-language evaluation initiatives in the US and Asia, working on other sets of languages. The final goal is to assist and stimulate the development of European cross-language retrieval systems in order to guarantee their competitiveness on the global marketplace.

Their site at the University of Neuchatel contains a great list of resources for natural language processing like stemmers and stop words lists.

Tags: natural language processing, text data mining

Beppe Grillo and thousands of others Italians (including me) have bought a page on the International Herald Tribune to inform about and protest against the 23 members of the parliament that have been convicted of a variety of crimes (including corruption) and yet are allowed to sit in the Parliament.

Recently he has been interviewed by the BBC concerning the reasons of this protests. You can hear the interview here. Here is the list containing the 23 names of the member of the Parliament with the indication of their charges.

Tags: Italy, politics

M. Diligenti, F. M. Coetzee, S. Lawrence, C. L. Giles, and M. Gori. Focused crawling using context graphs. In Proceedings of the 26th VLDB Conference, pages 527–534, Cairo, Egypt, 2000. [pdf]

——————-

This article presents an approach to web crawling which is defined as focused as opposed to standard crawling. Standard crawling uses a technique called ‘forward crawling’, that is following the links found on a page to the nexts. This has the limit to not exploit the lower levels of the trees of a web site from the point of entry of the crawler.

The authors also list other limits of the standard crawling technique: a) the limited ability to sacrifice short term document retrieval gains in the interest of better crawl performance; b) the lack of learning strategies where topically relevant documents are found by following off-topic pages.

The authors’ contribution is the Context Focused Crawler, an engine that uses Google or Altavista to define the ‘backward crawling’ from a certain page. This is used to build a context graph of the page: a network of documents that moves from the target document backwards following the hyperlinks.

Afterwards these documents are used to learn some classifier using a reduced version of the TF/IDF algorithm. Subsequently, by computing a naive bayes likelihood function for the winning layer, and comparing this to a threshold, it is possible to discard weakly matching pages.

A side remark is that the authors uses the name Context Graph but this has not to be confused with the Ceglowski [2003] Contextual Network Graph. The reason is that the graph is not assembled in the same way using a Term/Document procedure and also that the retrieval function is not base on Spreading Activation technique.

Tags: google, information retrieval

KidsOK is a location web service that allows parents to track their kids. By sending an SMS to a control number the parents receive back an MMS with the piece of the map of the city where the kids phone is located or was located the last time.

Tags: Location Based Services