Email spider and automatic mailing required

I want to centralise a job that I have to continually maintain at present.

1. Currently I am searching for emails related to a specific industry using ‘super email spider’. (This has bugs in it however and if it crashes, you lose 24-48hrs information.)

2. Once you have enough emails, you export and import it into a filter program which removes email addresses which are incomplete or part of a filter list manually added over time.

3. This then is imported into Civicrm and a cronjob email is sent out.

I want a centralised php page where I can basically do the above in one and leave it to run.

Specifications I can think of but will need further discussion are;

1. Control the pace at which the spider crawls the web as not to hog server resources. Be able to change this on the fly.

2. As per super email spider, be able to limit the number of searches on a particular page/url before moving on to the next entry.

3. Human feature for the search engines to delay queries so not to be blocked.

4. Be able to search a mix of keywords or sites urls.

5. Be able to add keywords on the fly and it updates the search without having to restart the process. i.e. If you have an input box for keywords, whilst it is working on this list, if you add and save more to this list, then it will add them to the queue and continue working.

6. Be able to save/recover from a server crash from where it left off. I am anticipating this script running for months or more at a time, so this is quite vital.

7. Manage duplicate email addresses, only send once in a particular batch run.

8. Handle email opt out requirements.

9. Be able to add criteria to the filter system and again it update it whilst in flow with immediate effect. If something is added to the filter, it does not process the applicable email address.

10. An emailing threshold so not to again overload servers or be flagged by spam filters. Again change this on the fly if it is the middle of a batch run.

11. SMTP emailing to pass through the relevant account.

OTHER

There are many tools out there doing these jobs seperately, they just need pulling together. If my price estimate is too low, I will pay more if there is the technical requirement to do so.

PAYMENT

I'll only pay a deposit to people with previous feedback, otherwise I won't be paying a deposit to a stranger who has just registered. Once a working/partial demo is seen, then deposit/full payment can be made as agreed.

CONTACT

Please do not send me a one liner saying 'we can do this job' as they will simply be deleted. If you can't write a simple email (ideally with previous work/references) then I don't have much faith in you delivering the task.

FINAL

We are a media company and would like to build up a working relationship with a person/small company that can offer quality work at decent rates and timeframes so this project could be the start of more should things develop (parden the pun).

Thanks in advance.

Andy

MySQL PHP

Nº del proyecto: #832200

Sobre el proyecto