Monday, December 28, 2020

Sia Not a Fit for Personal Backups

In this post, I will describe my experience trying to use Sia as personal backup service. Sia is a decentralized cloud storage platform.  I heard about Sia back in late 2017 when just about everyone was talking about cryptocurrencies, altcoins, and blockchains.  It appears to still be active 3 years later, so I took that as a good sign.  Spoiler alert - I got everything working, but learned that Sia is not a good fit for a personal backup service.

Alternative to CrashPlan

Ever since CrashPlan changed their focus to enterprise users and raise their price to a minimum of $10/month, I have been looking for a lower-cost file backup alternative for my home computer.  I finally decided to give Sia a try. If it worked out, I could reduce my annual backup expense from $120 to just $5!

Before I get into my Sia experience, here is an overview of all the steps I will attempt:
  1. Read about Sia (30 minutes)
  2. Download Sia UI (5 minutes)
  3. Wait for blockchain syncing to complete (8 hours)
  4. Purchase Siacoin in Bittrex (30 minutes).  Note, if you don't already have an account with a crypto exchange that supports Siacoin, then this step might take up to 24 hours to get set up.
  5. Transfer Siacoin to Sia UI wallet (60 minutes)
  6. Download SiaSync (10 minutes)
  7. Sync files (3 hours)
  8. Verify ability to restore on a different computer (4 hours)

Sia Terminology

I did a little reading on Sia's website to get familiar with their terminology.  I am a Renter that will be uploading files to other computers owned by other people called Hosts.  For that privilege, I will be configuring an Allowance of Siacoin (SIA), which is how much money I'm willing to spend.

The Sia UI

My home computer is an Ubuntu 18.04 desktop.  I downloaded the SIA UI Linux application from their Getting Started page. Double clicking it didn't do anything.  I made it executable and then launched it from the command line:

chmod +x Sia-UI-v1.5.3.AppImage
./Sia-UI-v1.5.3.AppImage

This brought up the SIA UI.  It assigned me a Sia Seed which I saved as a secure LastPass note.   The app displayed a message saying it was syncing and I could see the Block Height continue to increase, but I didn't know what that meant.  I selected Rent from the menu and then Setup Allowance.  This launched a calculator where I plugged in my estimated initial usage of 0.1 TB (100 GB).  It said the target price was 1250 SC which is the cost of a TB per month.  I wasn't sure if that is the current market rate or some kind of bid that I will be placing, but I kept it as-is, and continued.  Next I was asked if I am sure I want to set an allowance of 250 SC to rent storage on Sia.  I confirmed and was then given an error message saying that I have to wait for Sia to finish syncing before I can set an allowance.  I actually tried this several times because at first it seemed like nothing happened.  Later I found out that all of these contracts were queuing up and would go into effect as soon as my SC arrived in my wallet.  I wish the Sia UI had shown me these pending contracts so I wouldn't have generated so many!  My Block Height had reached over 40,000 by this point, but there was no indication of how long I'd have to wait for syncing to complete.

Sia UI

The Big Wait

I found this support post Why does Sia take so long to sync? which explained that before getting started, Sia would have to download an entire copy of its blockchain (over 22 GB) which could take several hours to over a day!  They should really tell you this when you first open the UI.  I decided to leave this to finish syncing over night and come back in the morning.  According to SiaStats, the Block Height needed to get to 292,652.  Sure enough, it had completed syncing by the morning.  The Dashboard showed 292,652 Block Height, 9 Connected Peers, and 203 Active Hosts.

With the Sync completed, I selected Rent in the UI and tried to set up an Allowance. I received a message saying Allowance successfully updated.  However, nothing in the Rent or Wallet section updated, so I have a feeling nothing actually happened since I don't have any Siacoin yet.

Getting Siacoin

The next step was to obtain some Siacoin (SC) I could use to purchase a contract.  Luckily I already had a Bittrex account which is one of the exchanges where you can trade for Siacoin.  Siacoin was currently trading at 1 SC for 0.00000013 BTC.  I tried to buy 1,500 SC for 0.00019568 BTC ($5.30 USD) which I estimated would last me a whole year.  Bittrex warned me that the minimum trade was 0.0005 BTC.  So I increased to 4,500 SC for 0.00058704 BTC ($15.91 USD) and tried again.

Bittrex Limit Order of SC

Transferring Siacoin

Once the order went through, I had to transfer the SC out of Bittrex and into my Sia UI Wallet.  Within the Sia UI, I went to Wallet, then Receive, then Generate New Address.  I initiated a Withdrawal within Bittrex, and entered the wallet address that I just generated from Sia UI.  The withdrawal showed as Completed in Bittrex pretty quickly.  However, I had to wait about 7 minutes before it showed up in the Sia UI.  I was relieved it showed up, though, as cryptocurrency transfers generally make me nervous.  The transaction status showed as 0/6, and I had to wait another 10 minutes before transaction fully completed with status of 6/6.  I transferred 1,500 SC, but was charged a 0.1 SC transaction fee, so 1499.9 SC ended up in my Sia UI wallet.

Contracts

Looks like 50 Contracts got created and and 143.52 SC were allocated when I attempted to set up an Allowance several times earlier.  I still don't really understand Contracts well enough to know how many I should have.  For now I'm going to assume that I have more than what I need and move on.

Syncing Files

To set up file syncing between my desktop and the Sia network, I turned to Siasync.  As new files are created or removed from a source folder, it will add or remove them from the Sia network accordingly.  This replicates the behavior I had with CrashPlan and what we're all used to with services like Dropbox.  Following their instructions, I downloaded the Siasync binaries for the latest version, 1.0.1 beta 1 and made the file executable from the command line:

chmod +x Siasync-linux-amd64

I was hoping I could make a single folder containing symlinks to other folders I wanted to sync.  I tried it, but Siasync wasn't recognizing any symlinked folders.  To sync multiple folders as subfolders to a single backup folder on the Sia network, I used the following separate commands:

./Siasync-linux-amd64 -sync-only -size-only -subfolder backups/Desktop ~/Desktop
./Siasync-linux-amd64 -sync-only -size-only -subfolder backups/Music ~/Music
./Siasync-linux-amd64 -sync-only -size-only -subfolder backups/Pictures ~/Pictures
./Siasync-linux-amd64 -sync-only -size-only -debug -subfolder backups/Videos ~/Videos

This worked, and all my files got backed up on the Sia network!

While searching for solutions on syncing using symlinks, I came across this Sia Cloud Storage Extension for GNOME which promises to take the place of Siasync with support for symlinks that you can put in a synced ~/Sia folder.  I made a note to try that later. 

Sanity Check

As a last step, I wanted to check if I could access my files from another computer. To do this, I downloaded Sia on a Macbook Pro. To try to save some time, I copied the consensus.db file (all 22 GB of it) from my Ubuntu desktop (192.168.0.51) so Sia UI wouldn't have to download it again:

scp ken@192.168.0.51:.config/Sia-UI/sia/consensus/consensus.db ~/Library/Application Support/Sia-UI/sia/consensus/.

This only took a few minutes, but Sia UI was still taking hours to finish configuring itself.

While waiting, I checked back with the Sia UI on my Ubuntu desktop and saw that the file uploads appeared to be stuck.   The health of each file was still at 0% and the number of items uploading wasn't changing.  I read lots of Sia documentation, but I couldn't figure out if something was wrong or if I just had to wait longer.  I found that Sia had a Discord server, so I signed on there to try to get help from the community.  I posted about my problem in the #help channel, and a nice person named Covalent helped me out.  He asked me to show him the output of siac renter -v.  After seeing that he concluded that my allowance was way too small.  He also explained to me that each file would take up a minimum of 40 MB!  With over 35,000 files, I was going to have to pay for over 1.4 TB of data!  That would make my backups cost even more than I was paying for CrashPlan.  Sia docs mention this 40 MB atomic file size here, but I didn't see anything about this in the Getting Started docs.  They should really make this clear for people up front.  If I knew that going into it, I wouldn't have done any of this.

Final Thoughts

It's been a long 24 hours learning about Sia and getting it set up as a personal backup system. In the end, it just doesn't seem well-suited for this use case, at least not in an affordable way.  I might try zipping up my directories and storing them on Sia as fewer, but larger compressed files. If that proves to be too convenient, I'll be sticking with CrashPlan for the time being.



Sunday, September 06, 2020

Simplifying Custom Events for Google Analytics using Google Tag Manager

I am relatively new to using Google Tag Manager (GTM), and I am trying to figure out the best way to log Custom Events from a single page app built with React.  I want these events to register in Google Analytics (GA) which I have set up already in Google Tag Manager.  The kinds of events I'm planning to record are user account creation, email validation, user sign in, user sign out, and various other interactions within the application. It looks like there are various ways to trigger events based on form submissions using Google Tag Manager, but I thought it would just be simpler to manually "invoke" each event I want exactly where I want it via the GTM API.

Searching the web, this was the most useful tutorial I found: Google Tag Manager Custom Event Trigger Explained by Julius Fedorovicius.  I made a spreadsheet for my app to capture all the custom events I am interested in.  There were 26 of them.  As I followed the tutorial, I noticed the following things I would need to do:

  • Create and push various Variables into the GTM Data Layer to use them in my event labels and values.
  • Create a new GTM Trigger for each of the 26 custom events from my spreadsheet
  • Create a new GTM GA Tag and associate the corresponding trigger for each of the 26 custom events from my spreadsheet
This seemed like a lot of work and maintenance within GTM.  I wondered if there was an easier way.  Since it is possible to define the event's Category, Action, Label, and Value using variables from the Data Layer, I realized that should be able to handle all of my 26 events with just 1 new GTM Tag and Trigger along with a reusable set of custom Variables.  Here's how I set it up:

Step 1: Create Data Layer Variables for each possible GA Event field


Google Tag Manager Custom Data Layer Variables for Google Analytics Event













This will be used for all the possible event fields Category, Action, Label (optional), and Value (optional) to be captured from the Data Layer.

Step 2: Create a Custom Event Trigger


Google Tag Manager Custom Event Trigger















This is meant to be generic so it can be used for any Custom Event.

Step 3: Create a Custom Event Tag and Associate with Custom Event Trigger


Google Tag Manager Custom Event Google Analytics Tag





























This Google Analytics tag will be triggered by the Custom Events.  Each attribute of the GA tag (Category, Action, Label, and Value) should be configured to come from the corresponding Data Layer Variable that was created in Step 1.  This is what allows the same Trigger and Tag to be used for all Events.  That's the whole set up!

The list of Tags now only includes one Tag for Custom Events instead of one for each Custom Event.  The tag list should look something like this:

Google Tag Manager Tag List










Pushing Events to Google Tag Manager

Now within the application, Custom Events can be pushed to Google Tag Manager's Data Layer and the Trigger that was set up will push them into Google Analytics.  The code would look something like this:

<script>
 window.dataLayer = window.dataLayer || [];
 window.dataLayer.push({
 'event': 'Custom Event',
 'eventCategory': 'My Event Category',
 'eventAction': 'My Event Action',
 'eventLabel': 'My Event Label',
 'eventValue': 0
 });
</script>


Final Thoughts

Perhaps the downside of this approach is that it may be harder to adapt these events to other analytics tags I may use in the future.  Maybe that is why Julius Fedorovicius's post referenced above suggested creating a new Trigger and Tag for each individual event.  I was thinking about trying out Mixpanel next, so I will soon find out.  For all you Google Tag Manager and Google Analytics Experts out there, please let me know your thoughts.  

Monday, June 22, 2020

Dividing Money

I have found a few situations that require taking an amount of money and dividing it up as evenly as possible, to the nearest cent or whole dollar, such that the individual parts exactly add up to the original amount.

Here are a few examples of when I've needed to do this:

Example 1:

Apportioning an Operating Expense to Multiple Real Estate Properties

As a real estate investor that owns multiple properties, I sometimes have operating expenses that are not associated with any individual property.  Since the Schedule E tax form requires all expenses to be listed on a per-property basis, it is necessary to take the expense and divide it up across the properties.  As an example, let's say I spent $1025.95 on tax preparation and I have 7 properties.  I would split that expense as follows:

 Property Amount
 A $146.56
 B $146.56
 C $146.56
 D $146.56
 E $146.57
 F $146.57
 G $146.57
 Total: $1025.95

Notice that in this case I use a precision of cents.

Example 2:

Distributing Cash Tips at a Gig

As a band leader, at the end of a gig, I often need to collect the cash tips we received, total them up, and split them with the rest of my band.  For example, if my jazz trio earns a total of $113 in tips, I would pay each musician as follows:

 Musician Amount
 myself $37
 bass player $38
 drummer $38
 Total: $113

In this case, I use a precision of whole dollars since we typically don't receive any coins.

These are just 2 examples, but similar division logic can be used when friends need to split the bill at a group dinner and other similar situations and the amount of money cannot be split evenly.  It's not rocket science to make these calculations, but I do it often enough that I decided to make a simple web-based tool to do it for me.  The tool can be found at:


In this tool, simply enter an amount of money, how many parts you want to divide it into, and the precision (cents or whole dollars), and then click Divide to get the resulting amounts. Feel free to use it for all your money splitting needs!  And let me know what you think of it in the comments below.