80 Commits

Author SHA1 Message Date
dandds
ca1d26cc67 Give the cluster perms to create load balancers.
In order for the cluster app registration to create new load balancers,
it needs to have the Network Contributor role for the virtual network.
In the future, we should create a custom policy scoped to exactly the
permissions the cluster needs, per:

https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni#prerequisites
2020-01-30 16:28:02 -05:00
Rob Gil
181d0155b7 Removes the vnet gateway since we're no longer going to use Azure VPN services 2020-01-30 14:43:59 -05:00
Rob Gil
b0a73e5944 Adds log analytics logging to redis module 2020-01-30 14:43:59 -05:00
Rob Gil
211a9e7875 Adds log analytics logging to cdn module 2020-01-30 14:43:59 -05:00
Rob Gil
ffb29d45fc Adds Redis logging to log analytics 2020-01-30 14:43:59 -05:00
Rob Gil
308ea6a861 Enables logging for storage account buckets
There are bugs in the latest version of the azurerm provider which do
not account for all the log types. We had to use a local-exec here.
2020-01-30 14:43:59 -05:00
Rob Gil
8475b519ce Adds log analytics logging for k8s 2020-01-30 14:43:59 -05:00
Rob Gil
27f3096593 Adds log analytics configuration to postgres 2020-01-30 14:43:52 -05:00
Rob Gil
8856d0f448 Log FMT 2020-01-30 14:43:07 -05:00
Rob Gil
b61bb6a4c4 Configures KeyVault to log to a log analytics workspace 2020-01-30 14:43:07 -05:00
Rob Gil
3e4244fc6d Configures container registry to log to a log analytics workspace 2020-01-30 14:43:07 -05:00
Rob Gil
0900c01d88 Adds log_analytics module and workspace 2020-01-30 14:43:07 -05:00
tomdds
0bc0e15134 Merge branch 'staging' into azure-config-values 2020-01-29 16:50:44 -05:00
dandds
adacb6ff19 Cleanup cruft 2020-01-27 13:17:09 -05:00
dandds
058ee57527 Create database with separate script.
Creating the ATAT database requires a separate connection to one of the
default Postgres databases, like `postgres`. This updates the scripts
and secrets-tool command to handle creating the database. It also
removes database creation from Terraform and updates the documentation.
2020-01-27 13:17:09 -05:00
dandds
a8f6befc17 secrets-tool command for bootstrapping database.
This additional secrets-tool command can be used to run the database
bootsrapping script (`script/database_setup.py`) inside an ATAT docker
container against the Azure database. It sources the necessary keys from
Key Vault.
2020-01-27 13:17:09 -05:00
Rob Gil
76465e978a Remove k8s test tf 2020-01-24 07:36:24 -05:00
Rob Gil
7b2523254d Adds Dans home ip 2020-01-24 07:36:02 -05:00
Rob Gil
daa07f8631 Removes unnecessary locals in the bucket module 2020-01-23 20:26:27 -05:00
Rob Gil
e0d59eb166 Finally fixes subnet list output
This finally fixes the output coming from the vpc module so that it
returns a full list of subnets. Now they can be referenced just like the
redis module is using in this commit.
2020-01-23 20:22:53 -05:00
Rob Gil
3f5bbf2c5e Cleans out comments 2020-01-23 19:58:06 -05:00
Rob Gil
9f0904c201 Adds dedicated redis subnet 2020-01-23 19:57:45 -05:00
Rob Gil
0f5f5bd926 Converts redis to use service_endpoints
This is still a WIP.
2020-01-23 19:16:00 -05:00
Rob Gil
38ce1ef2b2 Adds list of users for access to storage and more service endpoints
This sets up the rest of the service endpoints on the subnets. It also
adds a variable map specifically to grant IP access to the storage
buckets. This new variable map is necessary since the azure storage ip
rules do not accept /32 CIDR ranges. The rest of the services do support
cidr ranges.
2020-01-23 18:41:29 -05:00
Rob Gil
536eccdb90 Container registry private networking and bucket cidr range fix 2020-01-23 13:13:56 -05:00
Rob Gil
dab6cdb7dc Locks down keyvaults to subnets and administrator ip addresses 2020-01-23 11:02:12 -05:00
Rob Gil
c31d68a18c Makes client vpn cidr range configurable 2020-01-23 10:50:16 -05:00
Rob Gil
48482785ac Adds IP whitelisting to storage buckets 2020-01-23 10:02:31 -05:00
Rob Gil
d22357e609 Adds step to manually configure MFA in AD 2020-01-22 19:37:04 -05:00
Rob Gil
635ccb0fd3 Fixes postgres character collation 2020-01-22 19:36:33 -05:00
Rob Gil
01703b1488 Configures storage buckets to be optionally exposed via service endpoints 2020-01-22 19:35:54 -05:00
Rob Gil
9042a960bb Adds configurable service endpoints to subnets in the vpc module 2020-01-22 19:35:19 -05:00
dandds
83de5d38d9 Terraform initial database.
This addes TF config for creating the initial database for the selected
ATAT environment. The datatase name format is [environment]-atat.
2020-01-21 19:52:09 -05:00
dandds
9f2bdd4a9f Updated dev environment for JEDI.
- Updated environment name.
- Updated variables.
- AKS service principal creds moved to the operator Key Vault.
2020-01-21 10:08:27 -05:00
dandds
fdd8e3dbba
Merge pull request #1340 from robgil-dds/additional-tf-docs
Additional quick steps on how to configure terraform
2020-01-20 19:27:10 -05:00
Rob Gil
584b885311 Adds notes on AKS service_principal and preview features that must be enabled 2020-01-20 16:10:55 -05:00
Rob Gil
636653a5ad Additional quick steps on how to configure terraform 2020-01-20 15:37:01 -05:00
Rob Gil
4eded23051 Adds keyvault outputs 2020-01-20 14:04:51 -05:00
dandds
ad70042774
Merge pull request #1333 from robgil-dds/170237669-updated-deploy-docs
Document process for adding secrets for redis and storage to keyvault…
2020-01-19 10:54:12 -05:00
Rob Gil
9684b608d4 Document process for adding secrets for redis and storage to keyvault with secrets-tool 2020-01-17 14:16:06 -05:00
Rob Gil
9c429e35da 170237669 - Converts postgres secrets to use keyvault
This changes the configuration of the postgres master username and
password. Instead of committing to source (short term hack), this now
sources those secrets from KeyVault. Those secrets are generated and
populated via secrets-tool.
2020-01-17 13:30:00 -05:00
Rob Gil
7aaad51f81 170268346 - Enables access for admins to manage certificates in keyvault
Grants access to the admins to manage certificates in keyvault
2020-01-17 09:28:42 -05:00
Rob Gil
55623028df Adds a secrets generator and loader
secrets-tool now has a feature to both generate secrets as well as load
the generated secrets in to KeyVault.
2020-01-16 21:40:26 -05:00
Rob Gil
b9a7efe6ba Revised Pipfiles 2020-01-16 18:19:33 -05:00
Rob Gil
aa89505650 169163334 - Abstracts terraform wrapper code
The terraform wrapper is now abstracted in to a utility class for
working with terraform. The terraform module was also updated to support
configurable keyvault servers. Logging for this new module was also
added, so the terraform output is seen on the console.
2020-01-16 17:27:49 -05:00
Rob Gil
deead852b5 169163334 - Initial secrets-tool commit
Adds admin_users map and keyvault policy

This adds an admin_users map as well as a new policy in the keyvault
module. When run, this will apply an administrator policy for users in
the admin_users map. With these permissions, the admin users will be
able to manage secrets and keys in keyvault.

169163334 - Initial secrets-tool commit

Adds admin_users map and keyvault policy

This adds an admin_users map as well as a new policy in the keyvault
module. When run, this will apply an administrator policy for users in
the admin_users map. With these permissions, the admin users will be
able to manage secrets and keys in keyvault.

170237669 - Makes the read only policy for keyvault optional and only create the policy if a principal_id is passed

170237669 - Adds new operator keyvault for secrets

This is a new keyvault specifically for storing operator secrets and
things that would not be accessible to applications. The primary use
case for this is for launching things like postgres (root postgres
creds) and other services which would require secrets to be added to the
terraform configuration. This approach avoids adding secrets to
terraform.

An accompanying script will be added to populate the new keyvault.
2020-01-16 17:27:49 -05:00
Rob Gil
a47ad24b90 Apply manual change for dev environment to gain access to keyvault from vmss nodes 2020-01-13 12:08:09 -05:00
Rob Gil
8416c18258 Document keyvault post terraform manual steps
This is to document and configure the post-terraform commands necessary
for k8s hosts in the vmss to access the keyvault through flexvol.
2020-01-13 12:05:52 -05:00
Rob Gil
53cf42103e Fix resource names for module.vpc 2020-01-13 10:29:12 -05:00
Rob Gil
316428a787 Adds screenshot of manual change to make for SystemAssigned identities 2020-01-13 10:29:12 -05:00