websiteid – Identifying logical Web sites by flow simulation
Introduction
websiteid is a small set of Java classes written during my DEA (master's) internship at INRIA in 2003, under the supervision of Serge Abiteboul: a multithreaded crawler that extracts the link graph of a Web server, and the identification of logical Web sites in this graph by a flow simulation. Clustering of the graph relies on van Dongen's MCL, which is not included. The method is described in the following publications:
- Website Identification (master's thesis, 2003);
- Identifying Websites with Flow Simulation (ICWE 2005), and its longer version.
Download
The code is provided as is, as it was left in 2003, for research purposes.