fr.paris.telecomParisTech.dbWEB.arcomem.wp5.appliocationAwareHelper.crawling
Class CommonUtil

java.lang.Object
  fr.paris.telecomParisTech.dbWEB.arcomem.wp5.appliocationAwareHelper.crawling.CommonUtil

public class CommonUtil
extends Object

Extract the relevant crawling action from KB using a relavant XPath expression.

Field Summary

AppLevelHandler

instanceme

Constructor Summary

CommonUtil()

Method Summary

boolean	checkExistObject(String check, ArrayList nameArray) Check if object is already extracted.
ArrayList	copy_KB_detect_rules(String exp, ArrayList mylist) Copy the detection rules of all Content Management System in the ArrayList
ArrayList	copy_KB_pred_rules(String exp, ArrayList mylist) Copy the Predicates relevant to the sub query from the predicate file.
boolean	detectPageLevel(String expression)
String	detectWebApplication(String Queryexpression, ArrayList stringlist) Check one by one all detected rules for Web application Detection
String	extractTitle(String exp) Extract the title of the Web page.
void	getcrawlingaction(String exp) Get the Relevant crawling Action from KB.
ArrayList	getoutlinks(String reg, ArrayList collectinglinksWithAnchor) This will get the set of links with anchor text from current Web page.
ArrayList	getsetcrawlingaction(String exp) Extract related crawling actions for given XPath expression.
ArrayList	getUnWAoutlinks(String reg, ArrayList collectinglinksWithAnchor) Extracts links with anchor
ArrayList	Load_Extract_Objects_KB(String objects) Extracts the Objects name with xpath expression from KB.
int	runXpathcheckpage(String exp) Check whether it is level or not.
int	runXpathdetectApp(String xpathexp) Detect Web application against Expression.
void	runXpathdetectAppname(String exp) Set the Application Name of Detected Content Management System
ArrayList	runXpathdetectlevelRules(String exp, ArrayList mylist) Copy the Specific level (Forum or Thread or post) detection rules into dynamic arrays.
void	runXpathdetectlevelRules(String level, String exp) Copy the Specific level (Forum or Thread or post) detection rules into dynamic arrays

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

instanceme

public AppLevelHandler instanceme

Constructor Detail

CommonUtil

public CommonUtil()

Method Detail

checkExistObject

public boolean checkExistObject(String check,
                                ArrayList nameArray)

Check if object is already extracted.

Parameters:

check -

nameArray -

Returns:

true if alrady extracted else false

copy_KB_detect_rules

public ArrayList copy_KB_detect_rules(String exp,
                                      ArrayList mylist)
                               throws ParserConfigurationException,
                                      SAXException,
                                      IOException,
                                      XPathExpressionException

Copy the detection rules of all Content Management System in the ArrayList

Parameters:

exp - expression for copying

mylist - copy the result

Returns:

Array of relevant predicates

Throws:

XPathExpressionException

ParserConfigurationException

SAXException

IOException

copy_KB_pred_rules

public ArrayList copy_KB_pred_rules(String exp,
                                    ArrayList mylist)
                             throws ParserConfigurationException,
                                    SAXException,
                                    IOException,
                                    XPathExpressionException

Copy the Predicates relevant to the sub query from the predicate file.

Parameters:

exp - expression for copying

mylist - copy the result

Returns:

Array of relevant predicates

Throws:

XPathExpressionException

ParserConfigurationException

SAXException

IOException

detectPageLevel

public boolean detectPageLevel(String expression)
                        throws XPathExpressionException,
                               ParserConfigurationException,
                               SAXException,
                               IOException,
                               org.htmlcleaner.XPatherException

Throws:

XPathExpressionException

ParserConfigurationException

SAXException

IOException

org.htmlcleaner.XPatherException

detectWebApplication

public String detectWebApplication(String Queryexpression,
                                   ArrayList stringlist)
                            throws XPathExpressionException,
                                   ParserConfigurationException,
                                   IOException,
                                   org.htmlcleaner.XPatherException,
                                   SAXException

Check one by one all detected rules for Web application Detection

Parameters:

Queryexpression - Expression to be evaluated

stringlist - List of detected rules

Returns:

Id of Content Management System

Throws:

XPathExpressionException

ParserConfigurationException

IOException

org.htmlcleaner.XPatherException

SAXException

extractTitle

public String extractTitle(String exp)
                    throws ParserConfigurationException,
                           SAXException,
                           IOException,
                           XPathExpressionException,
                           org.htmlcleaner.XPatherException

Extract the title of the Web page.

Parameters:

exp - Xpath expression

Returns:

Title of the Web page.

Throws:

ParserConfigurationException

SAXException

IOException

XPathExpressionException

org.htmlcleaner.XPatherException

getcrawlingaction

public void getcrawlingaction(String exp)
                       throws ParserConfigurationException,
                              SAXException,
                              IOException,
                              XPathExpressionException

Get the Relevant crawling Action from KB.

Parameters:

exp - XPath Expression

Throws:

ParserConfigurationException

SAXException

IOException

XPathExpressionException

getoutlinks

public ArrayList getoutlinks(String reg,
                             ArrayList collectinglinksWithAnchor)
                      throws ParserConfigurationException,
                             IOException,
                             org.htmlcleaner.XPatherException,
                             XPathExpressionException,
                             SAXException

This will get the set of links with anchor text from current Web page.

Parameters:

reg - expression

collectinglinksWithAnchor - collection of Anchor and Link

Returns:

Double Array List with anchor text and outLink

Throws:

ParserConfigurationException

IOException

org.htmlcleaner.XPatherException

XPathExpressionException

SAXException

getsetcrawlingaction

public ArrayList getsetcrawlingaction(String exp)
                               throws ParserConfigurationException,
                                      SAXException,
                                      IOException,
                                      XPathExpressionException

Extract related crawling actions for given XPath expression.

Parameters:

exp - XPath expression.

Returns:

set of Crawling actions.

Throws:

ParserConfigurationException

SAXException

IOException

XPathExpressionException

getUnWAoutlinks

public ArrayList getUnWAoutlinks(String reg,
                                 ArrayList collectinglinksWithAnchor)
                          throws ParserConfigurationException,
                                 IOException,
                                 org.htmlcleaner.XPatherException,
                                 XPathExpressionException,
                                 SAXException

Extracts links with anchor

Parameters:

reg - expression to extract links

collectinglinksWithAnchor - collection box for links with anchor

Returns:

return that box

Throws:

ParserConfigurationException

IOException

org.htmlcleaner.XPatherException

XPathExpressionException

SAXException

Load_Extract_Objects_KB

public ArrayList Load_Extract_Objects_KB(String objects)
                                  throws XPathExpressionException,
                                         ParserConfigurationException,
                                         SAXException,
                                         IOException

Extracts the Objects name with xpath expression from KB.

Parameters:

objects - Expression to run over KB for extracting object expression

Returns:

load the Object extraction exppression from KB into Array

Throws:

ParserConfigurationException

SAXException

IOException

XPathExpressionException

org.htmlcleaner.XPatherException

runXpathcheckpage

public int runXpathcheckpage(String exp)
                      throws ParserConfigurationException,
                             SAXException,
                             IOException,
                             XPathExpressionException,
                             org.htmlcleaner.XPatherException

Check whether it is level or not.

Parameters:

exp - Xpath expression

Returns:

1 if found else 0

Throws:

ParserConfigurationException

SAXException

IOException

XPathExpressionException

org.htmlcleaner.XPatherException

runXpathdetectApp

public int runXpathdetectApp(String xpathexp)
                      throws ParserConfigurationException,
                             IOException,
                             org.htmlcleaner.XPatherException,
                             XPathExpressionException,
                             SAXException

Detect Web application against Expression.

Parameters:

xpathexp - Expression

Returns:

1 if detected else 0

Throws:

ParserConfigurationException

IOException

org.htmlcleaner.XPatherException

XPathExpressionException

SAXException

runXpathdetectAppname

public void runXpathdetectAppname(String exp)
                           throws ParserConfigurationException,
                                  SAXException,
                                  IOException,
                                  XPathExpressionException

Set the Application Name of Detected Content Management System

Parameters:

exp - XPath expression

Throws:

ParserConfigurationException

SAXException

IOException

XPathExpressionException

runXpathdetectlevelRules

public ArrayList runXpathdetectlevelRules(String exp,
                                          ArrayList mylist)
                                   throws ParserConfigurationException,
                                          SAXException,
                                          IOException,
                                          XPathExpressionException

Copy the Specific level (Forum or Thread or post) detection rules into dynamic arrays.

Parameters:

exp - XPath expression for copying

mylist - copy the result

Returns:

Array of relevant predicates

Throws:

XPathExpressionException

ParserConfigurationException

SAXException

IOException

runXpathdetectlevelRules

public void runXpathdetectlevelRules(String level,
                                     String exp)
                              throws ParserConfigurationException,
                                     SAXException,
                                     IOException,
                                     XPathExpressionException

Copy the Specific level (Forum or Thread or post) detection rules into dynamic arrays

Parameters:

level - Level of web application

exp - copy expression for copying

Throws:

XPathExpressionException

ParserConfigurationException

SAXException

IOException