Scrapy Jobs

I need a robust yet straightforward solution that pulls only the text content from several e-commerce websites. The target fields are product names, long and short descriptions, category labels, and any text-based specifications; I do not need images or pricing data. The scraper should: • Handle pagination, dynamic or lazy-loaded sections, and common anti-bot measures without overloading the servers. • Output clean, well-structured data (CSV or JSON preferred) ready for import into my internal system. • Be written in readable, well-commented code—Python with Scrapy, BeautifulSoup, or Selenium is ideal, but I’m open to equivalent approaches if they achieve the same reliability. • Include simple configuration so I can add or swap target domains later....

I need a reliable solution that automatically pulls public-facing text from specific social-media websites and delivers the content back to me in a clean, structured format ready for downstream analysis. The task covers three steps: building or customising a scraper in Python that navigates the chosen platforms, capturing posts, comments and any associated meta-information I specify; cleaning the raw output to remove emojis, markup, duplicate lines and other noise; and packaging the final, deduplicated dataset as a CSV or JSON file. Please write the code so I can rerun it anytime (command-line script or Jupyter notebook is fine) and include concise setup instructions plus brief inline documentation. I expect respectful rate-limit handling and compliance with each platform’s public-d...

I'm seeking an experienced Python developer to help with data analysis and automate data scraping tasks. Key Requirements: - Develop Python scripts for data analysis. - Automate data scraping from websites. - Deliver clean, structured datasets for analysis. Ideal Skills: - Proficiency in Python, especially for data manipulation. - Experience with libraries like BeautifulSoup, Scrapy, or Selenium. - Strong background in data analysis, preferably with Pandas or NumPy. - Familiarity with data storage solutions (e.g., SQL, NoSQL). Looking for someone who can write efficient, reliable scripts and has a keen eye for detail.

I need a clean, one-time extraction of every registered agent listed on the Kerala RERA portal. The scope is limited to the publicly displayed agent name plus all available contact details—phone numbers, email addresses, and office addresses. No licence-status fields or property listings are required. Any stack is fine—Python (BeautifulSoup, Scrapy, Selenium), Node.js, or a headless browser workflow—as long as it handles pagination, hidden rows, or JavaScript-rendered tables and respects polite scraping practices. Deliverables • An Excel workbook (.xlsx) containing one row per agent and clearly labelled columns for Name, Phone, Email, and Address • Data fully deduplicated, UTF-8 compliant, and free of blank placeholders • A short note on the appro...

I need two separate, non-blocking crawlers—one targeting Amazon, the other Flipkart—each operating with our seller accounts. Every 24 hours the system must fetch fresh data for the ASINs I supply: product name, seller name, star rating, ratings count, MRP, and current selling price. The information should be written straight into a MySQL database and the entire cycle has to repeat indefinitely without manual intervention. Reliability is critical. If either crawler encounters captchas, rate limits, network drops, or any other interruption, it should automatically recover (self-healing logic) and continue from the exact point of failure. Should a fault persist beyond the retry logic, I want an immediate email alert with a concise error report so I can act quickly. Please struct...

I need a robust web-scraping solution that automatically collects product information from several e-commerce websites. The focus is on two key data points: • Product name and full description • Customer reviews and ratings Price and availability are not required this time, so the crawler can ignore any endpoints related to stock or cost. Please build the script so I can run it on demand and easily point it at new store URLs in the future. Python with BeautifulSoup, Scrapy, or a similar framework suits me fine, as long as the code is clean, well-commented, and leverages polite scraping practices (respectful delays, user-agent rotation, handling captchas when possible). Deliverables: 1. Working scraper code with clear setup instructions 2. Sample CSV or JSON export c...

Project Title: Large-Scale Course Data Scraping from Udemy, Coursera, and YouTube Project Overview: I am looking for an experienced data scraping specialist who can collect a large dataset of online courses and tutorials from platforms such as Udemy, Coursera, and YouTube. The dataset should include both technical and non-technical tutorials. The objective is to build a structured dataset that can be used for research and analytics purposes. Project Requirements: The freelancer will scrape approximately 200,000 course/tutorial records from the following platforms: - Udemy - Coursera - YouTube The collected dataset should contain the following information for each record: 1. Course / Tutorial Title 2. Course URL / Video URL 3. Platform Name (Udemy / Coursera / YouTube) 4...

We need a robust yet lightweight script that can automatically pull business details from a publicly accessible government website. The information to capture will centre on business details, such as registration numbers, business name, registration date, address, etc. The workflow should: • Navigate every relevant section of the site (pagination, search filters, subsidiary pages). • Extract the required fields accurately • Export clean, structured data to CSV and JSON A Python solution leveraging requests/BeautifulSoup or Scrapy is preferred, but I’m open to other dependable stacks if they handle rate-limits, retries, and potential CAPTCHA gracefully. The script must be easy to rerun on demand, with clear instructions for environment setup and any dependencies...

Seeking a skilled web designer to create a clean, minimalistic website to showcase services. The website's primary function will be to highlight our offerings in a sleek, digestible format that engages visitors and inspires a sense of professionalism and quality. Features needed include: - A contact form to facilitate customer inquiries - Subscriber sign-up option for customers to receive updates and newsletters - A blog/news section to post updates and relevant content - An 'add listing' feature to display products effectively - Banner slide out et - Social Scroll It's crucial for the designer to have a keen eye for creating layouts that promote readability and user experience. Experience in minimalistic design can use template. Images & website examples will be...