samuel
/
cronpy
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2 Commits 1 Branch 0 Tags 91 KiB
Python 100%
Go to file
Samuel Campos 4e895f9297 Update readme.md 2020-10-03 21:40:35 +00:00
core Initial commit 2020-10-03 23:17:53 +02:00
doc Initial commit 2020-10-03 23:17:53 +02:00
lib Initial commit 2020-10-03 23:17:53 +02:00
providers Initial commit 2020-10-03 23:17:53 +02:00
scripts Initial commit 2020-10-03 23:17:53 +02:00
.gitignore Initial commit 2020-10-03 23:17:53 +02:00
aconvert_images.py Initial commit 2020-10-03 23:17:53 +02:00
create_schedule.py Initial commit 2020-10-03 23:17:53 +02:00
depopulate_news.py Initial commit 2020-10-03 23:17:53 +02:00
minify.py Initial commit 2020-10-03 23:17:53 +02:00
readme.md Update readme.md 2020-10-03 21:40:35 +00:00
setting.py Initial commit 2020-10-03 23:17:53 +02:00
update_admin_bets.py Initial commit 2020-10-03 23:17:53 +02:00
update_images.py Initial commit 2020-10-03 23:17:53 +02:00
update_month.py Initial commit 2020-10-03 23:17:53 +02:00
update_news.py Initial commit 2020-10-03 23:17:53 +02:00
update_notifications.py Initial commit 2020-10-03 23:17:53 +02:00
update_players.py Initial commit 2020-10-03 23:17:53 +02:00
update_preprod.py Initial commit 2020-10-03 23:17:53 +02:00
update_rankings.py Initial commit 2020-10-03 23:17:53 +02:00
update_schedule.py Initial commit 2020-10-03 23:17:53 +02:00
update_scores.py Initial commit 2020-10-03 23:17:53 +02:00
update_staff.py Initial commit 2020-10-03 23:17:53 +02:00
update_tvchannels.py Initial commit 2020-10-03 23:17:53 +02:00
update_tvschedule.py Initial commit 2020-10-03 23:17:53 +02:00
worker.py Initial commit 2020-10-03 23:17:53 +02:00

readme.md

1- INTRODUCTION

CronPY is a project made in Python to scrap different types of data about sports, such as :

  • all news about sports
  • schedules and scores
  • team staff
  • details about players
  • tv schedule

For that, different technologies have been used :

  • Python3.6 (with argparse, aiohttp, asyncio, beautifulsoup, selenium)
  • Redis (used for multi-process locks)
  • Mysql (used to store data)
  • InfluxDB (used to store details about each program execution)

Several websites are stored in order to gather different types of data :

  • eurosport.fr
  • rugbyrama.fr
  • fftt.com
  • footao.tv
  • football.fr
  • football365.fr
  • football-direct.com
  • footmercato.net
  • lequipe.fr
  • matchendirect.fr
  • programme-television.org
  • transfermarkt.fr

All these data are collected in nonprofit purpose for bestofbets.net, a website made for free sports predictions between friends. This is not maintained as I have started a new website to replace it : 1bet.fr (made with Python Django Framework and PostgreSQL). I decline any responsibility about your eventual usages of this project.

2- DEPLOYMENT

The deployment is quite basic, go on your project directory and execute these commands :

python3.6 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r doc/requirements.txt
deactivate
mkdir sam_aiohttp
cp -r venv/lib/python3.6/site-packages/aiohttp/* sam_aiohttp/
patch -p0 < doc/aiohttp.diff

MySQL, InfluxDB and Redis databases are needed for this program, as well as some environment variables, all clearly listed in setting.py.