-
-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leaks #6322
Comments
Is it possible that the backend isn't closing connections? What is If you could please collect a memory and goroutine profile that would help answer some questions: https://caddyserver.com/docs/profiling Thank you |
@mholt Here are memory and goroutine profiles here's an example of memory graph on the machine I got these profiles from |
Thanks -- interesting... what kind of traffic is your server getting? The top heaps and goroutines are simply growing buffers to read from TLS connections, and serving HTTP/2, respectively. Goroutines:
Heaps:
There's no memory leak here... just normal traffic. Not even a lot of traffic. What does I'd also be interested if you can reproduce this without the API backend. |
@mholt can this imply that my app doesn't close connections? I'll get back to you with |
Quite possibly. It's a bit unfortunate that the graphs only show a single process. That's why I'm curious what the output of |
@mholt, hello. The app is a java jetty server that has idle timeout of 200000ms (about 3.5 minutes) (doc) Here's a graph for the java app, grafana and caddy processes for the last month. As you can see, there was a restart after a certain memory threshold. The java app doesn't seem to constantly grow in memory. The java app receives mobile traffic, that probably can drop due to unstable internet connection As I can see in the caddy docs, caddy's default request idle timeout is 5 minutes |
What are your clients / traffic patterns like? Try setting some timeouts: https://caddyserver.com/docs/caddyfile/options#timeouts The majority of the (few) goroutines that show up in the profiles are waiting for reads. |
I'm not sure whether this can be related. On my memory usage graphs you can also notice that java memory sometimes is growing somewhat similarly (not here though!) Right now I'm also observing a memory leak within java |
Interesting... one other thought I have is, what if you disable HTTP/3? |
Has anyone been able to replicate this problem? |
I have a minimal config, and I constantly observe leaking memory. I used version 2.5.0 and later upgrated to 2.7, but this didn't help
Caddy is deployed via docker compose, here's a snippet:
The text was updated successfully, but these errors were encountered: