I am looking for help in designing an individual web-crawler and search script/utility for my site.
general information:
Registered users to my site have websites of their own. I want to offer an area where they can enter their URL into a field, and the script will crawl the user's url and save the crawled sites text to a text file or within a database. A basic search function for searching the text files will be needed as well - and search results should be displayed in a format like google (URL) X# of words.
Some statistics will be kept after the site text is crawled of all text, and those numbers will be used in how high results are displayed. Those numbers include..... 1) how many people click on the result. 2) how many times my keywords show in the text. 3) how many times a registered user logs into the main site (their results will show higher). 4) vote will be kept on result of 1-10 of how relevant..... and it will help show result higher.
That is the general idea, and ideas are welcome. there is a lot of free code on the web I can use, but I am looking for an individual who can add some creative thought. If you are part of a company looking for projects, I am not interested in your bids as I have had many problems and prefer individuals.
budget is about 400usd