Hi Kasper Laudrup,
Thanks for your reply, I have done scraper part for a single site and
performing well. But
Few points I want to share with you ,for that I need help.
1. Need to create web API for the listing of news and searching news with
keywords
2. Need to write a small scraper for below JSON links and get the news on
one API call for current
date and store it in the table in this format (Id, Date, Title, Details,
New From (CNN, Times,
etc.) and URL of News)
3. You need to ensure that no duplicate news gets entered in the table via
scraper
4. News can be searched via any keyword in title or in detail field
5. Also, we will need API to delete any news by passing ID and we need
separate API to update
the news details
{
"Washington Post": {
"link": "washingtonpost.com"
},
"CNN": {
"link": "http://edition.cnn.com/";
},
"Time": {
"link": "http://time.com/";
},
"QZ": {
"link": "https://qz.com/";
},
"Slashdot": {
"link": "https://slashdot.org/";
}
}
On Fri, Oct 9, 2020 at 11:48 PM Kasper Laudrup <laud...@stacktrace.dk>
wrote:
> Hi Viplay
>
> On 09/10/2020 16.26, Viplav Dube wrote:
> > Dear all,
> > Need to write a small scraper for JSON links and get the news on one API
> > call for current date and store it in the table in this format (Id,
> > Date, Title, Details, New From (CNN, Times, etc.) and URL of News).
> > News can be searched via any keyword in title or in detail field.
> >
>
> You can probably use the requests library
> (https://requests.readthedocs.io/en/master/) for the scraping.
>
> Not really sure if you mean you want to call an existing API, in which
> case just using requests should be fine, or you actually need to parse
> HTML and find links ("scraping") in which case you probably need to
> combine it with beatiful soup
> (https://www.crummy.com/software/BeautifulSoup/bs4/doc/).
>
> Shouldn't be a problem to create a database model for storing that
> information using standard Django models and searching shouldn't be much
> of a problem either:
>
> https://docs.djangoproject.com/en/3.1/topics/db/search/
>
> Hope that helps a bit.
>
> Kind regards,
>
> Kasper Laudrup
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/bb232ec7-672e-633f-1fb6-79c608693870%40stacktrace.dk
> .
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAGVfOwNVgaY4Spyugkb_jEPkcK%2Bd6N1zcHdb0rJjbQh48wtY1Q%40mail.gmail.com.
