Monday, December 28, 2020

Sia Not a Fit for Personal Backups

In this post, I will describe my experience trying to use Sia as personal backup service. Sia is a decentralized cloud storage platform.  I heard about Sia back in late 2017 when just about everyone was talking about cryptocurrencies, altcoins, and blockchains.  It appears to still be active 3 years later, so I took that as a good sign.  Spoiler alert - I got everything working, but learned that Sia is not a good fit for a personal backup service.

Alternative to CrashPlan

Ever since CrashPlan changed their focus to enterprise users and raise their price to a minimum of $10/month, I have been looking for a lower-cost file backup alternative for my home computer.  I finally decided to give Sia a try. If it worked out, I could reduce my annual backup expense from $120 to just $5!

Before I get into my Sia experience, here is an overview of all the steps I will attempt:
  1. Read about Sia (30 minutes)
  2. Download Sia UI (5 minutes)
  3. Wait for blockchain syncing to complete (8 hours)
  4. Purchase Siacoin in Bittrex (30 minutes).  Note, if you don't already have an account with a crypto exchange that supports Siacoin, then this step might take up to 24 hours to get set up.
  5. Transfer Siacoin to Sia UI wallet (60 minutes)
  6. Download SiaSync (10 minutes)
  7. Sync files (3 hours)
  8. Verify ability to restore on a different computer (4 hours)

Sia Terminology

I did a little reading on Sia's website to get familiar with their terminology.  I am a Renter that will be uploading files to other computers owned by other people called Hosts.  For that privilege, I will be configuring an Allowance of Siacoin (SIA), which is how much money I'm willing to spend.

The Sia UI

My home computer is an Ubuntu 18.04 desktop.  I downloaded the SIA UI Linux application from their Getting Started page. Double clicking it didn't do anything.  I made it executable and then launched it from the command line:

chmod +x Sia-UI-v1.5.3.AppImage

This brought up the SIA UI.  It assigned me a Sia Seed which I saved as a secure LastPass note.   The app displayed a message saying it was syncing and I could see the Block Height continue to increase, but I didn't know what that meant.  I selected Rent from the menu and then Setup Allowance.  This launched a calculator where I plugged in my estimated initial usage of 0.1 TB (100 GB).  It said the target price was 1250 SC which is the cost of a TB per month.  I wasn't sure if that is the current market rate or some kind of bid that I will be placing, but I kept it as-is, and continued.  Next I was asked if I am sure I want to set an allowance of 250 SC to rent storage on Sia.  I confirmed and was then given an error message saying that I have to wait for Sia to finish syncing before I can set an allowance.  I actually tried this several times because at first it seemed like nothing happened.  Later I found out that all of these contracts were queuing up and would go into effect as soon as my SC arrived in my wallet.  I wish the Sia UI had shown me these pending contracts so I wouldn't have generated so many!  My Block Height had reached over 40,000 by this point, but there was no indication of how long I'd have to wait for syncing to complete.

Sia UI

The Big Wait

I found this support post Why does Sia take so long to sync? which explained that before getting started, Sia would have to download an entire copy of its blockchain (over 22 GB) which could take several hours to over a day!  They should really tell you this when you first open the UI.  I decided to leave this to finish syncing over night and come back in the morning.  According to SiaStats, the Block Height needed to get to 292,652.  Sure enough, it had completed syncing by the morning.  The Dashboard showed 292,652 Block Height, 9 Connected Peers, and 203 Active Hosts.

With the Sync completed, I selected Rent in the UI and tried to set up an Allowance. I received a message saying Allowance successfully updated.  However, nothing in the Rent or Wallet section updated, so I have a feeling nothing actually happened since I don't have any Siacoin yet.

Getting Siacoin

The next step was to obtain some Siacoin (SC) I could use to purchase a contract.  Luckily I already had a Bittrex account which is one of the exchanges where you can trade for Siacoin.  Siacoin was currently trading at 1 SC for 0.00000013 BTC.  I tried to buy 1,500 SC for 0.00019568 BTC ($5.30 USD) which I estimated would last me a whole year.  Bittrex warned me that the minimum trade was 0.0005 BTC.  So I increased to 4,500 SC for 0.00058704 BTC ($15.91 USD) and tried again.

Bittrex Limit Order of SC

Transferring Siacoin

Once the order went through, I had to transfer the SC out of Bittrex and into my Sia UI Wallet.  Within the Sia UI, I went to Wallet, then Receive, then Generate New Address.  I initiated a Withdrawal within Bittrex, and entered the wallet address that I just generated from Sia UI.  The withdrawal showed as Completed in Bittrex pretty quickly.  However, I had to wait about 7 minutes before it showed up in the Sia UI.  I was relieved it showed up, though, as cryptocurrency transfers generally make me nervous.  The transaction status showed as 0/6, and I had to wait another 10 minutes before transaction fully completed with status of 6/6.  I transferred 1,500 SC, but was charged a 0.1 SC transaction fee, so 1499.9 SC ended up in my Sia UI wallet.


Looks like 50 Contracts got created and and 143.52 SC were allocated when I attempted to set up an Allowance several times earlier.  I still don't really understand Contracts well enough to know how many I should have.  For now I'm going to assume that I have more than what I need and move on.

Syncing Files

To set up file syncing between my desktop and the Sia network, I turned to Siasync.  As new files are created or removed from a source folder, it will add or remove them from the Sia network accordingly.  This replicates the behavior I had with CrashPlan and what we're all used to with services like Dropbox.  Following their instructions, I downloaded the Siasync binaries for the latest version, 1.0.1 beta 1 and made the file executable from the command line:

chmod +x Siasync-linux-amd64

I was hoping I could make a single folder containing symlinks to other folders I wanted to sync.  I tried it, but Siasync wasn't recognizing any symlinked folders.  To sync multiple folders as subfolders to a single backup folder on the Sia network, I used the following separate commands:

./Siasync-linux-amd64 -sync-only -size-only -subfolder backups/Desktop ~/Desktop
./Siasync-linux-amd64 -sync-only -size-only -subfolder backups/Music ~/Music
./Siasync-linux-amd64 -sync-only -size-only -subfolder backups/Pictures ~/Pictures
./Siasync-linux-amd64 -sync-only -size-only -debug -subfolder backups/Videos ~/Videos

This worked, and all my files got backed up on the Sia network!

While searching for solutions on syncing using symlinks, I came across this Sia Cloud Storage Extension for GNOME which promises to take the place of Siasync with support for symlinks that you can put in a synced ~/Sia folder.  I made a note to try that later. 

Sanity Check

As a last step, I wanted to check if I could access my files from another computer. To do this, I downloaded Sia on a Macbook Pro. To try to save some time, I copied the consensus.db file (all 22 GB of it) from my Ubuntu desktop ( so Sia UI wouldn't have to download it again:

scp ken@ ~/Library/Application Support/Sia-UI/sia/consensus/.

This only took a few minutes, but Sia UI was still taking hours to finish configuring itself.

While waiting, I checked back with the Sia UI on my Ubuntu desktop and saw that the file uploads appeared to be stuck.   The health of each file was still at 0% and the number of items uploading wasn't changing.  I read lots of Sia documentation, but I couldn't figure out if something was wrong or if I just had to wait longer.  I found that Sia had a Discord server, so I signed on there to try to get help from the community.  I posted about my problem in the #help channel, and a nice person named Covalent helped me out.  He asked me to show him the output of siac renter -v.  After seeing that he concluded that my allowance was way too small.  He also explained to me that each file would take up a minimum of 40 MB!  With over 35,000 files, I was going to have to pay for over 1.4 TB of data!  That would make my backups cost even more than I was paying for CrashPlan.  Sia docs mention this 40 MB atomic file size here, but I didn't see anything about this in the Getting Started docs.  They should really make this clear for people up front.  If I knew that going into it, I wouldn't have done any of this.

Final Thoughts

It's been a long 24 hours learning about Sia and getting it set up as a personal backup system. In the end, it just doesn't seem well-suited for this use case, at least not in an affordable way.  I might try zipping up my directories and storing them on Sia as fewer, but larger compressed files. If that proves to be too convenient, I'll be sticking with CrashPlan for the time being.


Anonymous said...

What about using something like duplication (40-50 MB chunks) or Borg backup (not sure if can work)?

Daniel Metcalfe said...

Great article that has given me important things to think about before considering this as a personal backup. I also appreciate your play-by-play style of the actual experience setting this up in detail, it agrees with my brain.

Thanks for the effort with this 🙏