41 lines
1.3 KiB
Markdown
41 lines
1.3 KiB
Markdown
## 1- INTRODUCTION
|
||
|
||
Scraper is a project made in Golang to scrap different types of data about sports, such as :
|
||
- all news about sports
|
||
- schedules and scores [coming soon]
|
||
- team staff [coming soon]
|
||
- details about players [coming soon]
|
||
- tv schedule [coming soon]
|
||
|
||
For that, different technologies have been used :
|
||
- `Golang` (with `go routines`, `goquery`, `gofeed`, `net/proxy`)
|
||
- `PostgreSQL` (used to store data)
|
||
- `InfluxDB` (used to store details about each program execution) [coming soon]
|
||
|
||
Several websites are stored in order to gather different types of data :
|
||
- eurosport.fr
|
||
- rugbyrama.fr
|
||
- fftt.com
|
||
- footao.tv [coming soon]
|
||
- football.fr [coming soon]
|
||
- football365.fr [coming soon]
|
||
- football-direct.com [coming soon]
|
||
- footmercato.net
|
||
- lequipe.fr
|
||
- matchendirect.fr [coming soon]
|
||
- programme-television.org [coming soon]
|
||
- transfermarkt.fr [coming soon]
|
||
|
||
All these data are collected in nonprofit purpose for `1bet.fr`, a website made for free sports
|
||
predictions between friends.
|
||
|
||
I decline any responsibility about your eventual usages of this project.
|
||
|
||
|
||
## 2- DEPLOYMENT
|
||
|
||
The deployment is very simple as the binary `scraper` can be used directly.
|
||
|
||
A PostgresSQL database is needed for this program, as well as some environment variables,
|
||
all clearly listed in `postgres.go`.
|