I’ve gotten just a little bit further with the decompression code 🙂
X-Message-Delivery: Vj0xLjE7dXM9MDtsPTA7YT0wO0Q9MTtTQ0w9MA==
X-Message-Status: n:0
X-SID-PRA: Daniel Parnell <me@danielparnell.com>
X-Message-Info: JGTYoYF78jHVrAq/T4xfKLLpTvwPwE0t6g/vyq6tyzbqln5iNnXwcSMg41wQCSulCxd5N8UB57Lq6un/ug7i0YsGX30yo+pm
Received: from randymail-a4.g.dreamhost.com ([208.97.132.207]) by bay0-mc8-f11.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.2668);
Thu, 29 Jan 2009 16:06:38 -0800
Received: from 192-168-1-101.tpgi.com.au (60-241-72-192.static.tpgi.com.au [60.241.72.192])
(using TLSv1 with cipher AES128-SHA (128/128 bits))
(No client certificate requested)
by randymail-a4.g.dreamhost.com (Postfix) with ESMTP id 97A67194DD3
for <danielparnell@live.com>; Thu, 29 Jan 2009 16:06:26 -0800 (PST)
Message-Id: <8EEE4310-25CC-47BE-8FCB-330EBCD75A21@danielparnell.com>
From: Daniel Parnell <me@danielparnell.com>
To: danielparnell@live.com
Content-Type: multipart/mixed; boundary=Apple-Mail-33–583770588
Mime-Version: 1.0 (Apple Message framework v930.3)
Subject: more test messages
Date: Fri, 30 Jan 2009 11:06:23 +1100
X-Mailer: Apple Mail (2.930.3)
Return-Path: me@danielparnell.com
X-OriginalArrivalTime: 30 Jan 2009 00:06:38.0411 (UTC) FILETIME=[9F1AB1B0:01C9826E]
The headers of the email message are coming through now, and I’m starting to get the message body
Hi Daniel! Thank for greate job! What kind of compression is used in Delta Sync?
Hi Mike,
the compression appears to be a non-standard variant of the LZ family from what I’ve been able to determine.
The files start with the prefix HU01, followed by a 4 bytes containing the length of the file header. This contains things like the uncompressed size and so on. After this header is the compressed data broken up into blocks 64K in size. Each block is prefixed with a magic number SCBH and again followed by 4 bytes containing the block header length. Immediately after this header is 256 bytes encoding the decompression table, and then following that is the compressed data for the block. The compression uses something like Huffman coding to store the compressed data. It also contains back references to already decompressed items.
It’s been very slow going as I don’t get a lot of time to work on it. I really should get around to writing up what I’ve discovered so far, if only to make it clearer for myself. I tend to get small pockets of time to work on it every couple of months and after that amount of time I have to relearn a lot of stuff 🙂
Hello:) Can you say more about protocol DeltaSync(signature, type compression, header HU01 and SCBH), give the full description of the DeltaSync protocol, please. Thank you:)
What dissasembler are you using? And what file are you working? I’nt find any function that may be used for compression.
I use IDA Pro. If anybody wants to donate a copy of the Hexrays decompiler….. 😉
Hi Daniel! Can you share with me the code you used to download the compressed messages from the servers? Thanks!
I’m currently working with the output of a packet capture. The code to download the messages is fairly simple compared to the decompression stuff.
Hey Daniel … looks like good work. Can you share your decompression code with me? Thanks!