1.6 KiB
1- INTRODUCTION
CronPY is a project made in Python to scrap different types of data about sports, such as :
- all news about sports
- schedules and scores
- team staff
- details about players
- tv schedule
For that, different technologies have been used :
Python3.6(withargparse,aiohttp,asyncio,beautifulsoup,selenium)Redis(used for multi-process locks)Mysql(used to store data)InfluxDB(used to store details about each program execution)
Several websites are stored in order to gather different types of data :
- eurosport.fr
- rugbyrama.fr
- fftt.com
- footao.tv
- football.fr
- football365.fr
- football-direct.com
- footmercato.net
- lequipe.fr
- matchendirect.fr
- programme-television.org
- transfermarkt.fr
All these data are collected in nonprofit purpose for bestofbets.net, a website made for free sports
predictions between friends. This is not maintained as I have started a new website to replace it : 1bet.fr
(made with Python Django Framework and PostgreSQL).
I decline any responsibility about your eventual usages of this project.
## 2- DEPLOYMENT
The deployment is quite basic, go on your project directory and execute these commands :
python3.6 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r doc/requirements.txt
deactivate
mkdir sam_aiohttp
cp -r venv/lib/python3.6/site-packages/aiohttp/* sam_aiohttp/
patch -p0 < doc/aiohttp.diff
MySQL, InfluxDB and Redis databases are needed for this program, as well as some environment variables,
all clearly listed in setting.py.