Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCLOUD: Support "private domains" plus many bugfixes #2482

Merged
merged 6 commits into from
Aug 8, 2023

Conversation

asn-iac
Copy link
Contributor

@asn-iac asn-iac commented Jul 13, 2023

Hi all,
First I'd like to acknowledge and thank the dnscontrol community for contributing to such a useful project.
We needed to implement some changes with the GCLOUD provider in a private fork of dnscontrol a few years back and finally have had time to submit some of these back to the public project.
This PR addresses a few existing bugs and adds some new features:

  1. Simplify GCLOUD authentication logic and remove dependency on separate http.Client (looks related to GCLOUD: Stop using deprecated New() call #1409
    // FIXME(tlim): Is it a problem that ctx is included with hc and in
    // the call to NewService? Seems redundant.
    )
  2. extend rate limit retry to additional API calls GCLOUD: add retry support to additional API calls #2478
  3. address an issue executing changes after a new zone is created via 'dnscontrol push' GCLOUD: inserting new zones in dnsconfig blocks changes on subsequent zones #2479
  4. add support for the creation of zones with "private" visibility GCLOUD: support creation of zones with private visibility #2481
  5. implement batching of corrections to resolve 403 error when executing corrections on zones w/ >1000 changes GCLOUD: Google will return an error if too many changes are specified in a single request  #2480

Happy to break these up into smaller PRs if needed.
TIA

@TomOnTime
Copy link
Collaborator

Wow! What a generous contribution!

I'm on vacation now but will get to this when I return.

@TomOnTime
Copy link
Collaborator

@riyadhalnur could you please review?

Copy link
Contributor

@riyadhalnur riyadhalnur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes 😃. I'll test this branch against real data and get back soon

return g, g.loadZoneInfo()
}

func (g *gcloudProvider) loadZoneInfo() error {
// TODO: In order to fully support split horizon domains within the same GCP project,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a good idea to create an issue for this for future action

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

created #2483

if key, ok := cfg["private_key"]; ok {
cfg["private_key"] = strings.Replace(key, "\\n", "\n", -1)
printer.Debugf("GCLOUD :private_key configured, attempting key-based authentication\n")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

if err != nil {
return nil, fmt.Errorf("no creds.json private_key found and ADC failed with:\n%s", err)
}
printer.Debugf("GCLOUD :using application default credentials\n")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

printer.Printf("GCLOUD :visibility %s configured\n", g.Visibility)
}
for i, v := range g.Networks {
networkURLCheck := regexp.MustCompile("^" + selfLinkBasePath + "[a-z][-a-z0-9]{4,28}[a-z0-9]/global/networks/[a-z]([-a-z0-9]{0,61}[a-z0-9])?$")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer to compile this once at the top rather than compiling it on every run of the loop

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved all of the regexp.MustComplie() calls to var block at the top - hope that was what you were looking for. let me know if you had something else in mind.

if networkURLCheck.MatchString(v) {
// the user specified a fully qualified network url
} else {
networkNameCheck := regexp.MustCompile("^[a-z]([-a-z0-9]{0,61}[a-z0-9])?$")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer to compile this once at the top rather than compiling it on every run of the loop

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved all of the regexp.MustComplie() calls to var block at the top - hope that was what you were looking for. let me know if you had something else in mind.

}
for i, v := range g.Networks {
networkURLCheck := regexp.MustCompile("^" + selfLinkBasePath + "[a-z][-a-z0-9]{4,28}[a-z0-9]/global/networks/[a-z]([-a-z0-9]{0,61}[a-z0-9])?$")
if networkURLCheck.MatchString(v) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this not be simpler to just continue if value is a fully qualified URL rather than an if-else statement

if ok := networkURLCheck.MatchString(v); ok {
    continue
}

if ok := networkNameCheck.MatchString(v); !ok {
    return nil, fmt.Errorf("GCLOUD :networks set but %s does not appear to be a valid network name or url", v)
}

...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated logic to remove if-else

return nil, err
}
if len(g.Visibility) != 0 {
visibilityCheck := regexp.MustCompile("^(public|private)$")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer to compile this once at the top

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved all of the regexp.MustComplie() calls to var block at the top - hope that was what you were looking for. let me know if you had something else in mind.

if len(g.Visibility) != 0 {
visibilityCheck := regexp.MustCompile("^(public|private)$")
if ok := visibilityCheck.MatchString(g.Visibility); !ok {
return nil, fmt.Errorf("GCLOUD :visibility set but does not match ^(public|private)$")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to say GCLOUD :visibility set but not one of public or private"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated message

Additions, Deletions []string
}

type orderedChanges struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The struct below duplicates this. We can just have one of them

@asn-iac
Copy link
Contributor Author

asn-iac commented Jul 14, 2023

Thanks team! I'm a bit tied up today but will review comments early next week

@tlimoncelli
Copy link
Contributor

Hi! I'm back from sabbatical/vacation!

Code looks great! Thank you so much for this contribution!

Merging soon.

@tlimoncelli tlimoncelli changed the title GCLOUD: features + bugfixes GCLOUD: Support "private" domains plus many bugfixes Aug 8, 2023
@tlimoncelli tlimoncelli changed the title GCLOUD: Support "private" domains plus many bugfixes GCLOUD: Support "private domains" plus many bugfixes Aug 8, 2023
@tlimoncelli tlimoncelli merged commit 1ea9f4c into StackExchange:master Aug 8, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants