GitLab

The overall workflow for this code base starts by grabbing a JSON list of account objects from the utility service /account/ GET endpoint at a regular interval. This JSON is parsed into a list of python dictionaries by a parser. Once parsed, the resultant python list of accounts should be passed to a function which will schedule the necessary celery tasks. This function, the scheduler, is considered deliverable when each of the following separate unit tests succeeds:

• Pass an empty list of accounts and check that no tasks are scheduled.
• Pass a list containing one new con edison account and check that one task is scheduled.
• Pass a list containing one new national grid account and check that one task is scheduled.
• Pass a list containing one previously-scraped con edison account and check that one task is scheduled.
• Pass a list containing one previously-scraped national grid account and check that one task is scheduled.
• Schedule a task to scrape a new account circumventing the scheduler function, pass the scheduler a list containing that account, then check that no new tasks are scheduled.
• Pass a list containing five accounts and check that five tasks are scheduled.

These tests will be run locally with a temporary local implementation of the data backends involved. When these tests are run with python -m unittest the entire scheduler data store (and any other persistent data) should be cleared between each test. When existence of a task is checked, it should also be confirmed that its arguments match those expected from the account.

To reiterate, this deliverable, the scheduler function, should assume a valid python list of account dictionaries as its only argument, should schedule a task for asap completion for each of the accounts, and should return nothing.

It will also be necessary to deploy this code to a production environment. As such, this deliverable will include documentation in the wiki associated with this repository for how to deploy the following onto an AWS EB environment:

• The message broker
• The result data store
• The scheduler