2020-10-03 21:17:53 +00:00
|
|
|
## 1- INTRODUCTION
|
|
|
|
|
|
|
|
|
|
CronPY is a project made in Python to scrap different types of data about sports, such as :
|
|
|
|
|
- all news about sports
|
|
|
|
|
- schedules and scores
|
|
|
|
|
- team staff
|
|
|
|
|
- details about players
|
|
|
|
|
- tv schedule
|
|
|
|
|
|
|
|
|
|
For that, different technologies have been used :
|
|
|
|
|
- `Python3.6` (with `argparse`, `aiohttp`, `asyncio`, `beautifulsoup`, `selenium`)
|
|
|
|
|
- `Redis` (used for multi-process locks)
|
|
|
|
|
- `Mysql` (used to store data)
|
|
|
|
|
- `InfluxDB` (used to store details about each program execution)
|
|
|
|
|
|
|
|
|
|
Several websites are stored in order to gather different types of data :
|
|
|
|
|
- eurosport.fr
|
|
|
|
|
- rugbyrama.fr
|
|
|
|
|
- fftt.com
|
|
|
|
|
- footao.tv
|
|
|
|
|
- football.fr
|
|
|
|
|
- football365.fr
|
|
|
|
|
- football-direct.com
|
|
|
|
|
- footmercato.net
|
|
|
|
|
- lequipe.fr
|
|
|
|
|
- matchendirect.fr
|
|
|
|
|
- programme-television.org
|
|
|
|
|
- transfermarkt.fr
|
|
|
|
|
|
|
|
|
|
All these data are collected in nonprofit purpose for `bestofbets.net`, a website made for free sports
|
|
|
|
|
predictions between friends. This is not maintained as I have started a new website to replace it : `1bet.fr`
|
|
|
|
|
(made with `Python Django Framework` and `PostgreSQL`).
|
|
|
|
|
I decline any responsibility about your eventual usages of this project.
|
|
|
|
|
|
|
|
|
|
|
2020-10-03 21:40:35 +00:00
|
|
|
## 2- DEPLOYMENT
|
2020-10-03 21:17:53 +00:00
|
|
|
|
|
|
|
|
The deployment is quite basic, go on your project directory and execute these commands :
|
|
|
|
|
|
|
|
|
|
python3.6 -m venv venv
|
|
|
|
|
source venv/bin/activate
|
|
|
|
|
pip install --upgrade pip
|
|
|
|
|
pip install -r doc/requirements.txt
|
|
|
|
|
deactivate
|
|
|
|
|
mkdir sam_aiohttp
|
|
|
|
|
cp -r venv/lib/python3.6/site-packages/aiohttp/* sam_aiohttp/
|
|
|
|
|
patch -p0 < doc/aiohttp.diff
|
|
|
|
|
|
|
|
|
|
MySQL, InfluxDB and Redis databases are needed for this program, as well as some environment variables,
|
|
|
|
|
all clearly listed in `setting.py`.
|
