![]() The learning curve can deter data scientists from taking on this type of work, which translates into a few (less than optimal) outcomes: You ask a separate team to do the job, or you just run it manually every time data is needed. ![]() While Airflow is the industry standard for building workflows, there is a lot to know before starting. The task of moving code developed locally and executed manually to a remote location that runs on a schedule can seem daunting at first.Ī standard recommendation for creating workflows is to learn Airflow. Save the resulting output to a database (e.g., Snowflake) or object storage (e.g., AWS S3).ĭespite the ubiquity of this process, I’ve noticed it is also one of the more challenging aspects for beginners to grasp.Query a database or scrape data from a website.The process is identical to the all-too-common Extract-Transform-Load (ETL) pattern we observe in data science: Still, I discovered a simple way to automate and schedule the execution of. Players with higher projections would “play,” while those with lower point projections would stay “on the bench.”Īfter comparing my model’s performance against Yahoo’s projections following the completion of the season, my projections were, as expected, close but less accurate (but that’s for a different discussion). The model predicted how many points each player on my roster would score in the upcoming week of games. Upon completion, the scraped data was fed into a predictive model. These inputs were part of a more extensive data-scraping process that ran every Wednesday. Instead, I wanted to see if I could come close to their point projections in terms of accuracy by creating a simple model with some essential inputs, like past player performance, betting lines, weather forecasts, and playing environment. My goal wasn’t to beat the “professional” fantasy football players, specifically Draft Kings or Yahoo. So this year, I decided to build a custom model to make my own weekly player point forecasts. While I’ve always used a “data-driven” approach to making roster decisions, I’ve predominantly relied on the forecasts and opinions of the internet. Fantasy Football season has sadly come to an end, so the next best thing I can do is…write about Fantasy Football! Indeed, this past season I decided to manage my team differently.
0 Comments
Leave a Reply. |