Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batched writes do not respect maxFileSize. #27

Open
kevinswiber opened this issue Jul 21, 2014 · 4 comments
Open

Batched writes do not respect maxFileSize. #27

kevinswiber opened this issue Jul 21, 2014 · 4 comments
Labels

Comments

@kevinswiber
Copy link
Member

The entire batch gets written to a single file.

@kevinswiber
Copy link
Member Author

Without managing an internal state of temp files, I'm not sure how to get around this. A batch bigger than the underlying operating system's maximum file size will break. If there's not a quick fix, I'm okay with this bug hanging out for a while.

@kesla
Copy link
Member

kesla commented Jul 22, 2014

perhaps one of the cases where we automatically should start a compaction (as disussed in #1) is when a batch has created a file larger than maxFileSize?

@kesla
Copy link
Member

kesla commented Jul 22, 2014

That way the batch would only create a single file larger than maxFileSize for a limited amount of time.

@kevinswiber
Copy link
Member Author

@kesla Yep. I tried auto-compacting on file wraps, too, but hit some concurrency error. I didn't dig into it. We just need a whole round of auto-compact commits (per #1).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants