This updates the configuration handling for the Redis connection string.
The motivation is so that the Redis password can be managed separately
via Azure Key Vault and eventually be rotated independently of the rest
of the connection URI.
This also tweaks the method we use to build the DATABASE_URI and removes
some stale config from the CI config file.
- modify Accordion component to be able to customize classes and tags
that it uses to build markup
- modify the shape of the object that the mock CSP returns for data that
builds this section.
- Also ensures that the CLINs TOs and CLINS are sorted
- add appropriate css for styling
In order to more easily sort JEDI Clins for the obligated funds section,
the JEDI CLIN enum values were used in the output dict instead of their
verbose labels. And in order to bring the labels in line with designs,
the JEDI CLIN labels were DRYed up in the translations file, which
required making small changes in a few other places across the project.
This commit lays out the genral structure and provides necessary
data for the new reporting page designs.
Some of the data generated by the report domain classes (including
the mock CSP reporting class) was modified to fit new designs. This also
included removing data that was no longer necessary. Part of the newly
mocked data includes the idea of "expended" data per CLIN or task order.
This was was mocked simply by using a 75% of the obligated funds fo a
given object. Tests were also written for these new/ modifed reporting
functions.
As for the front end, this commit only focuses on the high-level markup
layout. This includes splitting the large reporting index page into
smaller component templates for each of the major sections of the report.
1. Funding duration
Returns a tuple of the earliest period of performance start date and
latest period of performance end date for all active task order in a portfolio.
2. Days to funding expiration
Returns the numbei of days between today and the lastest
period performance end date of all active task orders
3. Active task orders
Returns a list of a portfolio's active task orders a
This enables JSON logging for Celery workers if the LOG_JSON conig value
is set. It uses the same JsonFormatter class used by the Flask
applications. That class has been updated in two ways:
- It takes a `source` kwarg to define the log source for the formatter.
- The `msg` attribute of the log record is formatted with any arguments
that may have been passed. This is necessary for Celery to render task
type, completion time, etc. into the log output.
Our content security policy in non-dev environments didn't allow uploading to azure blob storage. This adds a configurable blob storage base URL to allow regions to specify which storage endpoint they expect the upload request to use.
Since an invite that is expired has a status of 'pending', the ordering
of the if/elif statement needs to be switched so it checks to see if the
invite is expired before seeing if it is pending.
This value is set as the Access-Control-Allow-Origin header value for
the application. When using Azure CDN, the CDN will consume this header
when it populates its cache and use it on subsequent requests.
It would be possible to make this the same as the Flask SERVER_NAME
value. We explicitly set SERVER_NAME for Celery worker processes because
they need that information to contruct URLs outside of the request cycle
(Flask can infer the server name within a request cycle). I decided not
to rely on SERVER_NAME though because it has side effects:
- It determines what `url_for` uses as the host domain (which would be
fine).
- It makes it so that the Flask app can only server requests to that
domain (probably fine, but it felt like too big a side effect).
Additionally, SERVER_NAME does not include the scheme. For all of these
reasons I opted to make CDN_ORIGIN a separate config value.
disabled env role
We were only checking to see if a role was disabled or deleted before
raising an error, so I added in a check to see if the user was trying to
update the env role before raising an error. The error should only be
raised if the role is disabled or deleted AND the user is trying to
assign a new role to the env role.
I also added in a disabled property to the EnvironmentRole model to make
things more readable.
1. Prevents roles from being created with the role 'None'
2. Only call EnvironmentRoles.delete() if the env_role exists
3. Update the filter on the role field of the app member form to return
'No Access'. This fixed an issue where if a role was deleted, then other
env roles belonging to the app member could not be updated because the
role field of the deleted env_role was invalid
This change makes it so that when an env_role is updated to be None, the
role property on the env_role is changed to be None in addition to being
marked as deleted. This also adds in a check so that previously deleted
env_roles cannot be reassigned a role.
whitespace
The validator ListItemRequired() was only checking for None and an empty
string, not for strings that were multiple whitespace characters. This
fixes this issue by checking each item with regex to make sure it
contains non whitespace characters
The filter remove_empty_string() also was not checking for strings that
were multiple whitespace characters. This was also fixed by using regex
tomake sure that the string contains non whitespace characters, and also
clips any trailing whitespace.
The previous solution (ad-hoc stream-parsing the CRLs to obtain their
issuers and nextUpdate) was too cute. It began breaking on CRLs that had
an addition hex 0x30 byte somewhere in their header. I thought that 0x30
was a reserved character only to be used for tags in ASN1 encoded with
DER; turns out that's not true. Rather than write a full-fledged ASN1
stream-parser, the simplest solution is to just maintain the list of
issuers as a constant in the codebase. This is fine because the issuer
for a specific CRL URI should not change. If it does, we've probably got
bigger problems.
This also removes the Flask app's functionality for updating the local
CRL cache. This is being handled out-of-band by a Kubernetes CronJob
and is not a concern of the app's. This means that instances of the
CRLCache do not have to explicitly track expirations for CRLs.
Previously, the in-memory dictionary or CRL issuers and locations
included expirations; now it is flattened to not include that
information.
The CRLCache class has been updated to accept a crl_list kwargs so that
unit tests can provide their own alternative CRL lists, since we now
hard-code the expected CRLs and issuers. The nightly CRL check job has
been updated to check that the hard-coded list of issuers matches what
we get when we actually sync the CRLs.
In local development, the app will fail to start if it does not find the
directory specified by CRL_STORAGE_CONTAINER. This adds a few lines to
safely create that directory on startup and corresponding tests.
