Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement Tracking]: Improve & Fix some of the Ksctl internal code πŸ›£οΈ #334

Closed
30 tasks done
dipankardas011 opened this issue Apr 28, 2024 · 8 comments Β· Fixed by #342 or #370
Closed
30 tasks done
Assignees
Labels
kind/bug Something isn't working kind/enhancement New feature or request kind/feature priority/high highest priority triage/accepted

Comments

@dipankardas011
Copy link
Member

dipankardas011 commented Apr 28, 2024

Enhancement Title

Remove Unused components and fix some of them

Summary of the Enhancement

2024-05-15T10:12:09.9785281Z {"time":"2024-05-15T10:12:09.978109441Z","level":"WARN","msg":"RETRYING","component":"bootstrap","msgType":"WARN","reason":"dial tcp 131.153.226.231:22: connect: connection refused"}
2024-05-15T10:12:11.2170484Z {"time":"2024-05-15T10:12:11.216608586Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"moving the downloaded binaries to specific location"}
2024-05-15T10:12:12.1193148Z {"time":"2024-05-15T10:12:12.118966431Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"store the certificate files"}
2024-05-15T10:12:12.3327943Z {"time":"2024-05-15T10:12:12.332416862Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"configure etcd configuration file and systemd"}
2024-05-15T10:12:12.5085991Z {"time":"2024-05-15T10:12:12.508219611Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"restart the systemd and start etcd service"}
2024-05-15T10:12:13.0747725Z {"time":"2024-05-15T10:12:13.074287554Z","level":"WARN","msg":"RETRYING","component":"bootstrap","msgType":"WARN","reason":"ssh: handshake failed: read tcp 10.0.0.4:53054->131.153.227.53:22: read: connection reset by peer"}
2024-05-15T10:12:13.3066062Z {"time":"2024-05-15T10:12:13.306267622Z","level":"INFO","msg":"Successful in executing the script","component":"bootstrap","msgType":"SUCCESS"}
2024-05-15T10:12:13.3068565Z {"time":"2024-05-15T10:12:13.306359723Z","level":"INFO","msg":"configured DataStore","component":"bootstrap","msgType":"SUCCESS","number":"0"}
2024-05-15T10:12:20.9337979Z {"time":"2024-05-15T10:12:20.933498291Z","level":"INFO","msg":"Exec Scripts","component":"bootstrap"}
2024-05-15T10:12:20.9339935Z {"time":"2024-05-15T10:12:20.933524092Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"fetch etcd binaries and cleanup"}
2024-05-15T10:12:22.8695486Z {"time":"2024-05-15T10:12:22.869228268Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"moving the downloaded binaries to specific location"}
2024-05-15T10:12:24.0330621Z {"time":"2024-05-15T10:12:24.032757325Z","level":"INFO","msg":"Exec Scripts","component":"bootstrap"}
2024-05-15T10:12:24.0332581Z {"time":"2024-05-15T10:12:24.032787326Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"fetch etcd binaries and cleanup"}
2024-05-15T10:12:24.3713248Z {"time":"2024-05-15T10:12:24.370853956Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"store the certificate files"}
2024-05-15T10:12:25.9435970Z {"time":"2024-05-15T10:12:25.943199575Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"moving the downloaded binaries to specific location"}
2024-05-15T10:12:27.0220919Z {"time":"2024-05-15T10:12:27.021725771Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"store the certificate files"}
2024-05-15T10:12:34.1035786Z {"time":"2024-05-15T10:12:34.103211057Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"enable and start systemd service for haproxy"}
2024-05-15T10:12:34.9681396Z {"time":"2024-05-15T10:12:34.967829019Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"create haproxy configuration"}
2024-05-15T10:12:35.1401188Z {"time":"2024-05-15T10:12:35.139811418Z","level":"INFO","msg":"Executing Sub-Script","component":"bootstrap","name":"restarting haproxy"}
2024-05-15T10:12:35.4093997Z {"time":"2024-05-15T10:12:35.408584978Z","level":"INFO","msg":"Successful in executing the script","component":"bootstrap","msgType":"SUCCESS"}
2024-05-15T10:12:35.4096119Z {"time":"2024-05-15T10:12:35.408778781Z","level":"INFO","msg":"configured LoadBalancer","component":"bootstrap","msgType":"SUCCESS"}
2024-05-15T10:28:57.3958280Z {"time":"2024-05-15T10:28:57.395198366Z","level":"ERROR","msg":"handled error","catch":"Failure in executing script%!(EXTRA string=component, string=bootstrap, string=Reason, *errors.errorString=EOF, string=stderr, string=)"}
2024-05-15T10:28:57.3960989Z {"time":"2024-05-15T10:28:57.395294967Z","level":"INFO","msg":"Local Storage Got Killed","component":"local","msgType":"SUCCESS"}
2024-05-15T10:28:57.3964029Z {"time":"2024-05-15T10:28:57.395309367Z","level":"ERROR","msg":"Failure","err":"Failure in executing script%!(EXTRA string=component, string=bootstrap, string=Reason, *errors.errorString=EOF, string=stderr, string=)"}
2024-05-15T10:28:57.3975308Z exit status 1
# Copyright 2023 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
image
[PASS] Done adding kubernetes ksctl specific controllers
[PASS] Done with installing ksctl k8s specific tools
[ERR] handled error catch=App install failed component=kubernetes-client app=storage.Application{Name:"istio", Version:"latest"} Reason=failed to locate chart component=kubernetes-client Reason=no cached repo found. (try 'helm repo update'): open /Users/dipankardas/.cache/helm/repository/istio-index.yaml: no such file or directory


[PASS] Local Storage Got Killed
[ERR] Failure err=App install failed component=kubernetes-client app=storage.Application{Name:"istio", Version:"latest"} Reason=failed to locate chart component=kubernetes-client Reason=no cached repo found. (try 'helm repo update'): open /Users/dipankardas/.cache/helm/repository/istio-index.yaml: no such file or directory


exit status 1

Motivation and Goals

It will help enable to remove unwanted codebases

Detailed Design

Note

Document the changes and changelog as well

Alternatives

No response

Scope of changes

  • Cli
  • UI
  • Docs

Agreement

  • I agree to engage in constructive discussion and follow the Contribution Guidelines.
@dipankardas011 dipankardas011 changed the title [Enhancement Proposal]: Remove Depricated components [Enhancement Proposal]: UPdate components Apr 28, 2024
@dipankardas011 dipankardas011 changed the title [Enhancement Proposal]: UPdate components [Enhancement Proposal]: Update components Apr 28, 2024
@dipankardas011 dipankardas011 self-assigned this May 5, 2024
@dipankardas011 dipankardas011 linked a pull request May 9, 2024 that will close this issue
3 tasks
@dipankardas011
Copy link
Member Author

Caution

I am force all the context.TODO() and context.Background() already present to use the package specific context

cc @Horiodino

@dipankardas011 dipankardas011 changed the title [Enhancement Proposal]: Update components [Enhancement Tracking]: Update components May 12, 2024
@dipankardas011 dipankardas011 changed the title [Enhancement Tracking]: Update components [Enhancement Tracking]: Update components πŸ›£οΈ May 12, 2024
@dipankardas011
Copy link
Member Author

dipankardas011 commented May 14, 2024

Improve the loading screen like some go routine that we are waiting for some information

or wait operation

package main

import (
	"fmt"
	"time"

	"github.com/briandowns/spinner"
)

func main() {
	for i := 0; i <= 43; i++ {
		fmt.Println("i:", i)
		do(i)
	}
}

func do(i int) {
	s := spinner.New(
		spinner.CharSets[i],
		100*time.Millisecond,
		spinner.WithSuffix(" Waiting for Data to arrive"),
		spinner.WithFinalMSG("Done"),
		spinner.WithColor("green"))
	s.Start()
	time.Sleep(2 * time.Second)
	s.Stop()
}

@dipankardas011
Copy link
Member Author

dipankardas011 commented May 20, 2024

Some discussion on ksctl component release cycle

For example a new tag is created and a relase for ksctl core
new tag -> v1.2.0

Sol 1

in the package formed it will use this v1.2.0 version of the image only
some variable will be needed for storing each ksctl component version

Sol 2

if the ksctl apps manifests (deployment, services, pods, ...) are there in a specific loc

then we don't need to wait for them to be

func installApplication(struct{ Name: "ksctl-stateimport", Version: "v1.1.4" })

then we don't need to maintain these manifest specific to version

as it will be in speicif clocation
ksctl-components/manifests/**

@dipankardas011
Copy link
Member Author

dipankardas011 commented May 20, 2024

we would also need to think on for each release there has to be one file or something which will contain the latest stable release version for these components

and in predetermined location like the above mentioned

so proposal is ksctl-components/manifests/[state-import, agent, ...]/install.yml

and then we can provide tags to reach in specific branch

Warning

constraint: all the release must have this directory

@dipankardas011
Copy link
Member Author

dipankardas011 commented May 21, 2024

we can learn from this
image
image

@dipankardas011
Copy link
Member Author

dipankardas011 commented May 22, 2024

we can learn from this

Improved the generalLogger

@dipankardas011
Copy link
Member Author

dipankardas011 commented May 22, 2024

also try to make the implementations for mocks in the test cases only aka inside *_test.go

@dipankardas011
Copy link
Member Author

Need to investagate on this error

Bug civo managed

{"time":"2024-05-22T17:33:38.81591668Z","level":"INFO","msg":"Started to configure Cluster to add Ksctl specific storage","component":"kubernetes-client"}
{"time":"2024-05-22T17:33:38.815926202Z","level":"INFO","msg":"creating ksctl namespace","component":"kubernetes-client","name":"ksctl"}
{"time":"2024-05-22T17:33:39.06779658Z","level":"INFO","msg":"creating service account for ksctl agent","component":"kubernetes-client","name":"ksctl-sa"}
{"time":"2024-05-22T17:33:39.150461265Z","level":"INFO","msg":"creating clusterrole","component":"kubernetes-client","name":"ksctl-agent-crole"}
{"time":"2024-05-22T17:33:39.234801452Z","level":"INFO","msg":"creating clusterrolebinding","component":"kubernetes-client","name":"ksctl-agent-croleb"}
{"time":"2024-05-22T17:33:39.318191118Z","level":"INFO","msg":"creating ksctl state transfer pod","component":"kubernetes-client","name":"ksctl-state-importer"}
{"time":"2024-05-22T17:33:39.5035[68](https://github.com/ksctl/ksctl/actions/runs/9195349229/job/25291175559#step:4:69)483Z","level":"WARN","msg":"retrying current no of success","component":"kubernetes-client","msgType":"WARN","status.Phase":"Pending","status.Reason":""}
{"time":"2024-05-22T17:33:44.581762607Z","level":"WARN","msg":"retrying current no of success","component":"kubernetes-client","msgType":"WARN","status.Phase":"Pending","status.Reason":""}
{"time":"2024-05-22T17:34:29.58395594Z","level":"ERROR","msg":"handled error","catch":"pod get failed%!(EXTRA string=component, string=kubernetes-client, string=Reason, *url.Error=Get \"https://74.220.19.73:6443/api/v1/namespaces/ksctl/pods/ksctl-state-importer\": http2: client connection lost)"}
{"time":"2024-05-22T17:34:29.58402275Z","level":"INFO","msg":"Local Storage Got Killed","component":"local","msgType":"SUCCESS"}
{"time":"2024-05-22T17:34:29.584034494Z","level":"ERROR","msg":"Failure","err":"pod get failed%!(EXTRA string=component, string=kubernetes-client, string=Reason, *url.Error=Get \"https://74.220.19.[73](https://github.com/ksctl/ksctl/actions/runs/9195349229/job/25291175559#step:4:74):6443/api/v1/namespaces/ksctl/pods/ksctl-state-importer\": http2: client connection lost)"}

@dipankardas011 dipankardas011 changed the title [Enhancement Tracking]: Update components πŸ›£οΈ [Enhancement Tracking]: Improve & Fix some of the Ksctl internal code πŸ›£οΈ May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working kind/enhancement New feature or request kind/feature priority/high highest priority triage/accepted
Projects
None yet
1 participant