Skip to content
This repository was archived by the owner on Oct 16, 2024. It is now read-only.

DL-179: Support ZStandard Compression#106

Open
khurrumnasimm wants to merge 1 commit into
apache:masterfrom
khurrumnasimm:kn/zstd
Open

DL-179: Support ZStandard Compression#106
khurrumnasimm wants to merge 1 commit into
apache:masterfrom
khurrumnasimm:kn/zstd

Conversation

@khurrumnasimm

Copy link
Copy Markdown
Contributor

This PR is to add Zstd compression support.

@asfbot

asfbot commented Jan 6, 2017

Copy link
Copy Markdown

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/distributedlog-precommit-pullrequest/82/
--none--

@sijie

sijie commented Jan 7, 2017

Copy link
Copy Markdown
Member

@khurrumnasimm do you have any benchmark results to show performance? it seems we have only 2 bits to tolerant compression algorithm to the record/entry format. I'd like to confirm if the result is promising enough.

byte[] compressedData = getOrCopyData(data, offset, length);
int decompressedSize = (int) Zstd.decompressedSize(compressedData);
byte[] decompressedData = new byte[decompressedSize];
// TODO: make the compression level configurable.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

per DL-128, can we try to either tag such TODOs with jira tickets (or just avoid them if not necessary)?

@sijie

sijie commented Aug 15, 2017

Copy link
Copy Markdown
Member

It is an interesting work. I will try to pick up this one and carry it forward.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants