Sunday, April 28, 2019

Algorithms: Union-Find in C#

Union-Find Algorithm: Imagine if you have 100 computers in a room not networked to one another. You could use a Union(computer1, computer2) method to join 2 computers together in a network. From there, you could continue joining computers together randomly and you would want to have a method that returns a bool called Connected(computer1, computer2) that determines if they are already connected, so you don't have to do the work if they're already in the same network.
    /// 
    /// Initialize: N
    /// Union: N
    /// Find: 1
    /// Accesses: takes N^2 accesses to process sequence of N union commands on N objects
    /// Summary: Union is too expensive. Too slow for huge problems. Quick union will be a little faster. 
    /// 
    public class QuickFindUF
    {
        private int[] id;

        public QuickFindUF(int N)
        {
            id = new int[N];
            for (int i = 0; i < N; i++) id[i] = i;
        }

        public bool Connected(int p, int q) // Find
        {
            return id[p] == id[q];
        }

        public void Union(int p, int q)
        {
            int pid = id[p];
            int qid = id[q];

            for (int i = 0; i < id.Length; i++)
            {
                if (id[i] == pid) id[i] = qid;
            }
        }
    }
Now, if we wanted to be a little more efficient, we could setup the connections to be in a tree format. Each computer will reference it's parent's number as an ID. The root computers will reference themselves as their ID, so you can check for that to tell if a computer is a root of a tree.
    /// 
    /// Faster than QuickFindUF, but still too slow.
    /// Too expensive because trees can get too tall or too flat (extremes).
    /// 
    public class QuickUnionUF
    {
        private int[] id;

        public QuickUnionUF(int N)
        {
            id = new int[N];
            for (int i = 0; i < N; i++) id[i] = i;
        }

        private int Root(int i)
        {
            // Chase parent ID until reach the root (depth of i array accesses)
            while (i != id[i]) i = id[i];
            return i;
        }

        public bool Connected(int p, int q) // Find
        {
            return Root(p) == Root(q);
        }

        public void Union(int p, int q)
        {
            int i = Root(p);
            int j = Root(q);
            id[i] = j;
        }
    }
Lastly, we could have balanced trees to ensure we have no extremely tall trees. A component is how many groups of computers that we have a component could be 1 computer by itself, or it could be all of the computers in a component if every computer has been networked together at some point with the Union() method. We should keep track of the size of each component and how many total components we have in our data set.
/// 
    /// Keep track of tree size and take steps to avoid having tall trees.
    /// Smaller tree will connect to the root of the larger tree.
    /// Java implementation on p. 228
    /// Depth of any node x is at most lg N (lg = base-2 logarithms)
    /// If N is 1,000 its 10 (2^10 = 1000)
    /// If N is 1,000,000 its 20 (2^20)
    /// If N is 1,000,000,000 its 30 (2^30)
    /// Can also add path compression: just after computing root of p, 
    /// set the id of each examined node to point to that root
    /// 
    public class QuickUnionUF2 // Weighted Quick Union
    {
        private int[] id;
        private int[] sz;
        private int count; // number of components

        public QuickUnionUF2(int N)
        {
            id = new int[N];
            for (int i = 0; i < N; i++) id[i] = i;

            sz = new int[N];
            for (int i = 0; i < N; i++) sz[i] = 1;
        }

        public int Count() { return count;  }

        private int Root(int i)
        {
            // Chase parent ID until reach the root (depth of i array accesses)
            while (i != id[i])
            {
                // Halves path length. No reason not to, keeps tree almost completely flat.
                id[i] = id[id[i]]; // Only 1 extra line of code to do path compression improvement! 
                
                i = id[i];
            }
            return i;
        }

        public bool Connected(int p, int q) // Find
        {
            return Root(p) == Root(q);
        }

        public void Union(int p, int q)
        {
            int i = Root(p);
            int j = Root(q);

            if (i == j) return;

            // Make smaller root point to the larger root.
            if (sz[i] < sz[j]) // if i is smaller than j
            {
                id[i] = j; // point i to new root j
                sz[j] += sz[i]; // add the count of the smaller array to the count of the larger array
            }
            else // if j is smaller than i
            {
                id[j] = i;
                sz[i] += sz[j];
            }

            count--; // if we combine components, the count of components goes down by 1
        }
    }

Monday, March 4, 2019

Side Gigs while Transitioning Careers (or in general)

I've gotten this question a lot lately, so decided to just make a post about it: "What are side gigs I can do to earn money while transitioning careers?" I will list out all the ideas I can think of here and more details for the ones I know about.

Why do I know about these? I needed some of these when I switched from medicine to software and did some on the side on night/weekends.  I have friends with flexible work. I traveled a lot for work and talked to people around me. Sometimes I was on the customer side. And, reading PennyHoarder for kicks.

The list below will be mostly applicable to US residents. Some of this is available in other countries/regions, but I live in the US and can provide a list that is useful for where I live. If you are a good writer, don't mind driving, or are bilingual, you will have lots of options!

Remote:
Google Ads Rater - look at google search queries and see if the results are good, see if ads graphics are related to the text topic, see if queries for videos bring back a good result video, etc. To do this, basically apply for a spot within one of the below companies, wait for them to select you, you start taking exams from them based on material they send you (usually a large-ish PDF manual), pass tests, start doing work.  It can be a bit brain numbing but easy money around $13/hour that you can do while watching Netflix. It's not very intensive or hard as long as you are good at absorbing the rating requirements that were taught in the training manuals. Note historically there were 4 companies when I did this 5 years ago, it looks like in 2019 it is down to 2.
    • LionBridge - Current Job Openings (click on testers, raters, and curators for Google Ads rater jobs, if you are bilingual you can investigate the translator or interpreter options).
    • Appen Butler - Current Job Openings (click raters for google ads, or if interested checkout language jobs or micro tasks).
    • Leapforce has been acquired by Appen.
    • ZeroChaos <-- I believe was dropped from Google in 2017 due to this article.
VIPKID Teachers - teaching English via online conference call to children in China. Need to be very enthusiastic and require a Bachelor's degree in any field and to be in the US or Canada.
Amazon Turk - basically random tasks people need a person, not a robot to do. It could be literally anything manual, reading/writing/recording audio, analyzing, testing things out, checking grammar, translating, making subtitles, you name it. You can screw up and make like $1 an hour picking low paying tasks, or as much as $20ish an hour. I believe most folk average around $10 an hour. Do your research based on the links below and do your own research before embarking on this one so you know how to earn a decent hourly wage.
iTalki - If you can speak a foreign language and want to teach the foreign language, or teach English to those who know the same foreign language as you, this is a top notch website.  I used this site as a learner to learn a foreign language and its a reliable and well put-together process.  You sign up as a tutor, put up your schedule, and people sign up for lessons with you.  You do lessons via Skype video typically, unless you and the student agree on a different video conferencing program.  You can teach in any way you choose, come up with your own materials and plan based on the student.
  • Freelance Writing - I don't know much about this area but just giving this as an idea.
  • Translator - I don't know much about this area but just giving this as an idea.
Local In-Person Flexible Jobs:

  • Bartending - find a local bartending job at a hotel or restaurant.  The more expensive the joint, the better the tips will be.
  • Mystery Shopping -  You get paid to pretend to be a shopper and make sure employees are doing what they are supposed to (coffee at the right temperature, introductions, talk about some sales/rewards program, checking IDs for alcohol, etc.). Typically you get reimbursed for whatever item you buy and a payment for your time for doing the task and writing up a report.  You need to have decent grammar/writing/spelling skills and do a good job on the report.  If you mess up the report, you won't get paid or reimbursed, so be careful, but most people who are very meticulous will have no issues with this. Typically you pay for items with your credit card and are reimbursed, so you need a credit card and have to be ok with a reimbursement coming 7-90 days later.
  • Pet Sitting - you either visit/stay at someone else's place or take in their pet to your home.  There are websites that are local to your area and more national reach ones, search for pet sitter on google and you can checkout options for your area.
    • Rover - large customer/sitter base nationwide
  • Dog Walker - walk dogs! Google this and find out what you have locally, etc.
    • Rover - large customer/sitter base nationwide
  • Focus Groups - Usually target some specific demographic, if you are in it, apply! Mostly one and done, you apply, if you get in, you do some task, easy money, and they pay you right after.
  • Tutoring - tutor adults and kids in some topic! Could be elementary school math, SAT, MCAT, languages, anything! Do some research online for other options, the one I used was below:
  • Research Studies / Medical Research Trials - just an idea

Local In-Person Flexible Jobs Related to Driving and Food:

  • Uber - rideshare option, drive folk around on your own schedule. Most people join both Uber and Lyft to make sure they get rides from both. There are a lot of strategies to get the most rides, rush hour, weekend nights, and hovering around the airport are all good ideas.  You do often wait in a queue if you are in an airport or busy location like that.  Stick to downtown areas for weekend nights if you want the bar scene, also wrap up your car with plastic wrap in the back and provide doggy bags in case of sick folk who drank too much if you do this. 
  • InstaCart - food delivery or in-store shopper, sometimes the same person does both, but usually these are two separate jobs.
  • PrimeNow Shopper - work in Whole Foods and assemble orders
  • UberEats - food delivery
  • DoorDash - food delivery
  • GrubHub - food delivery
  • BiteSquad - food delivery
  • Postmates - food delivery
One last note, a pretty good website to come through for more ideas is The Penny Hoarder.

Thursday, January 17, 2019

How to Become a Business Analyst

Image result for business analyst

I created some training material for the IIBA (International Institute of Business Analysis) local Tampa chapter and it is open for anyone to utilize.  I have am working with them to help with training in using Azure DevOps (formerly VSTS/TFS) and for new folk to the field.

How to get into the field:
  • I would recommend getting the Scrum.org and the ECBA certification from the IIBA.  The best way to go for the ECBA certification is to pay for the membership for the IIBA which will give you lots of training materials and an exam discount (about the price of the membership so it's like free materials!).
  • Go to local meetups and search for IIBA in your area and find some nice folk to ask about the field and what the job is like!
  • When searching for jobs make sure to search for a generic "Analyst" as there are many, many names for Business Analysts.  Also add in keywords for "entry level" or "junior" and don't be afraid of the requirements or years of experience they ask for in the posting, that is simply a wish list of an ideal candidate, not a real person that they actually plan on finding (especially for junior roles). 
  • Write a cover letter explaining why you are interested in the Business Analyst role, why you are a good fit, what you have done to prepare yourself, what skills you have, and why you are interested in the particular company you are applying for.
  • On the top of your resume, list skills that are relevant to the IT field and the Business Analyst role.  Members of local meetups or industry Analysts can help you out!
  • Learn about Agile, Scrum, SDLC, Epics/Features/User Stories, and more by searching online.  Also... see below on this point:
I have some training on how to write Epics/Features/User Stories and examples of how to break things down.  I also have hands on labs on how to utilize Azure DevOps as a tool in the Business Analyst role.  Azure DevOps is completely free to use for up to 5 people (so for 1 person learning, it's a great tool). The labs and info is all free and I just hope it helps someone out!

Please click here to see it: https://github.com/catenn/IIBA/wiki

Note: I am not a Business Analyst, I am a developer who has partnered up with Business Analysts to make this training. I do work at Microsoft and I do train Business Analysts on how to utilize Azure DevOps as a tool, but I do not train Business Analysts on how to do their role.

Tuesday, January 15, 2019

Networking Part VI: Azure Networking - VMs, IP Addresses, and Load Balancing

This is part 6 of a series of networking posts:
Azure VM Networking
  • A NIC on a VM can connect to your VNet for example. Properties of a NIC that are normally set in the OS are able to be configured in Azure for your Azure VMs.  
    • Note: VMs (and all other resources) need to be deployed to the same region as your VNet.  So you need a VNet for each region that you have resources.
  • A NIC on a VM can also be attached to a load balancer (for highly available applications). 
  • Up to 250 IPs can be assigned per NIC, and these can be public or private IP addresses.  
    • You can add/edit IPs by going to your VM resource > Network interface > Settings > IP Configurations > Add.  If you select an existing IP address, you can go into another menu where you are able to enable it as Public/Private or assign it as Dynamic/Static. You can also change the actual IP address value for Static IPs.   Changing to a new IP address will cause the VM to reboot.
  • You can add more NICs to the VM if you need more IPs from a Microsoft block of reserved IP addresses (you cannot port in your own IP address ranges that you are using on prem or otherwise).
  • The underlying Azure platform handles connections to/from the machine, you should not RDP into your VM and setup a static IP address for you IPv4 (you will lose access to your machine, it is an unsupported action for an Azure VM and well documented). 
  • You also should not install a DHCP service on your VM (unsupported as well). You can do dynamic or static IP addresses for your VM.
    • Dynamic Host Configuration Protocol (DHCP) is a client/server protocol that automatically provides an Internet Protocol (IP) host with its IP address and other related configuration information such as the subnet mask and default gateway.
  • MAC address persistence: by default we guarantee that the MAC address will not change for a VM or NIC provided the VM is not deleted, redeployed from disks, or redeployed using the redeploy feature that rebuilds the virtual hardware.
  • When you create a new VM you can add it to an existing VNet and subnet.  If you want it publicly available, you can add Public IP, or if you don't then select None for the Public IP address (and it will automatically get assigned only a private IP address).  You should create a new NSG for it (default option).

IP addresses
  • IP addresses are treated as a separate resource in Azure, which allows you to move it from one resource to another.
  • Not all resources in Azure support a static (reserved) IP. For example, a VPN Gateway or Application Gateway do not support a static address because they are Microsoft managed resources. The dynamic IP will not change unless you delete or redeploy that resource, however. 
  • Static (reserved) Public IPs - retain the IP address.  The IPs can be moved between services easily and within seconds. 
  • Public IPv4 resources are a finite resource. You can have 5 public static IP addresses as part of your Azure subscription for no cost, but any additional ones will start becoming a billable resource. Dynamic IPs can help you with cost savings as they are cheaper than static IPs.
Load Balancing
Azure has 4 different options for traffic distribution:

1. Traffic Manager - DNS based
An illustration showing Azure Traffic Manager routing a user request to the nearest data center.

2. Azure Load Balancer - Layer 4 of OSI model (TCP/UDP)
An illustration showing the web tier of a three-tier architecture. The web tier has multiple virtual machines to service user requests. There is a load balancer that distributes user requests among the virtual machines.

3. Application Gateway - Layer 7 of OSI model (HTTP/HTTPS), can start to add in some Web Application Firewall (WAF), SSL Termination and Cookie based affinity
4. 3rd Party Virtual Appliances - Layer 3-7 of OSI model.  Network Virtual Appliances (NVAs) are simply custom IaaS VMs that can provide network functions such as firewall, app firewalls, IDS/IPS, load balancer, and VPN terminators. Azures IP forwarding and UDRs can help send packets to these VMs or override default routing behavior.  Many NVAs are available in the Azure Marketplace. Some common vendors are:

  • Cisco
  • Citrix
  • F5
  • Infoblox
  • Check Pint
  • Fortinet

Networking Part V: Azure Networking Foundations - VNet, Subnet, Security, and Connectivity

This is part 5 of a series of networking posts:
Azure Networking Overview
Azure data centers manage the physical hardware for you. You configure virtual networks through software, which enables you to treat a virtual network just like your own network. For example, you can choose which other networks your virtual network can reach, whether that's the public internet or other networks in the private IP address space.

Azure Foundation Networking
Image result for vnet subnet diagram

  • Virtual Network: logically isolated network on Azure that enables the flow of communication between different resources/subnets/VMs, other VNets, or to on-premises (depending on how you set it up). This is the foundation of Azure Networking and it is a Layer-3 overlay. Virtual networks are segmented into one or more subnets. Limitation of VNets: VNets cannot span regions or subscriptions.  VNet Peering, ExpressRoute, or VNet-to-VNet can connect regions or subscriptions together, however.
  • Address Space: usable IP addresses within your virtual network. Mainly an IPv4 environment. Public and private available.
      • VNets are isolated by default. If you deploy two or more VNets in the same address space, it is fine as long as you do not intend to ever try to connect them. 
      • If you need to add multiple address spaces, go to your VNet resource > Settings > Address Space and add more. As you add more, you can make subnets from all added address spaces.
      • Azure Reserved Addresses are needed for Azure core connectivity.  /29 is the smallest and /9 is the largest you can use in Azure. Typically x.0.0.4 will be the first usable address space. 
    • Subnet:  Subnets can help you organize and secure your resources.
      • In the Portal when you create a VNet it has  you define your first subnet, you can add more later  or delete/modify the one you created initially.
      • Network Security Group: controls ingress and egress traffic (allows or denies) to your Azure resources such as a NIC or subnet. Think of a network security group as a cloud-level firewall for your network. Prioritized set of rules based on a 5-tuple rule-set: source + destination IP, source + destination port, and protocol.  Can expose only certain ports of a subnet or NIC to the Internet as well as secure the flow of traffic between subnets/NICs on the same subnet.  Stateful rules and will keep track of your requests.
      • Route Tables: A route table contains a set of rules, called routes, that specifies how packets should be routed in a virtual network. Route tables are associated to subnets, and each packet leaving a subnet is handled based on the associated route table. Each route table can be associated to multiple subnets, but a subnet can only be associated to a single route table.
    • DNS Servers: References to DNS servers that will be assigned to the VMs or cloud server instances in the VNet. DNS resolution service is provided by default out of the box with your VNet in Azure. 
      • You can specify a custom DNS service
    • User Defined Routes (UDR): control traffic to a much finer degree. 
      • Send traffic to/from IPS or IDS for monitoring/auditing.
Security
1. Policies - Azure Policies can be setup for general security to ensure that your company/team follows rules that you setup.  Below are some good practices for networking specific policies:

  • Keep resources within a specific region
  • Prevent resources from being provision with Public IPs
  • Force resources location to match their resource group
  • Force certain users/prefixes to be deployed only to certain subnets (like dev/qa/prod networks)
  • Constrain the Azure regions that is allowed to be deployed to (whitelisting)

2. Setup RBAC for user access.
3. Setup a User Defined Route (UDR).
4. Setup a Network Security Group (NSG).

IPSec Tunnels
Internet Protocol Security (IPsec) is a secure network protocol suite that authenticates and encrypts the packets of data sent over an internet protocol network. It is used in virtual private networks (VPNs). In tunnel mode, the entire IP packet is encrypted and authenticated. It is then encapsulated into a new IP packet with a new IP header. Tunnel mode is used to create virtual private networks for network-to-network communications (e.g. between routers to link sites), host-to-network communications (e.g. remote user access) and host-to-host communications (e.g. private chat). - https://en.wikipedia.org/wiki/IPsec#Tunnel_mode

Connectivity Within Azure Options
  1. VNet Peering within Region - Connects two VNets in the same region via the backend network in Azure.  No extra overhead with this and almost no latency. 
  2. Global VNet peering - connecting VNets across Azure regions
  3. VNet to VNet via VPN Gateway - Leverages the Azure VPN gateway in each VNet to make the IPSec tunnel across regions or within the same region. Can also span to other customers or subscriptions easily.
  4. VNet to VNet via ExpressRoute - Associate multiple VNets to the same circuit.  Behind the scenes, Azure will enable routing between these VNets and their different regions to all connect to each other easily.  Could cause security concerns with default behavior, but can be controlled. 
Hybrid Connectivity from On Premises to Azure Options
  1. Internet Connectivity - resources are public facing directly to Internet and you can connect from anywhere.  While possible, not usually preferred by companies. 
  2. Secure point-to-site connectivity - IPSec based tunnel that based from a client machine. Really good for individuals traveling and outside of the corp network and need to access private resources. Individual developers can use this.
  3. Secure site-to-site VPN connectivity - Most enterprises start here, simple IPSec tunnel over public Internet connectivity from your on-premises gateway and Azure.  Connect to private IaaS and PaaS resources seamlessly, should just feel like an extension from your network. 
  4. ExpressRoute private connectivity - Enterprise solution geared towards customer who need high throughput (~10GB range) / low latency connections. Dedicated direct connection from your datacenter to Azure (pretend there's a big cable going from your datacenter to Azure datacenter).  Backed by SLA.

Monday, January 7, 2019

Azure Containers Offerings Comparison

Updates: This was last updated 1/7/2019. Azure Containers offerings are constantly changing and services are moving from PREVIEW to GA all the time.
  • *GA = Generally Available, backed by SLAs and guaranteed up-time. Meant for production workloads
  • *PREVIEW = available for beta use/early access use, but not backed by SLA or guaranteed up-time, not meant for production workloads yet
  • *SLA = Service Level Agreement
Note: Price estimates are for average workloads on the service and could be less than or exceed the estimate depending on how you utilize the service.

Service Ease of Use / Cost / OS Support Scalability General Use Case
Azure Container Instances (PaaS) Easiest and quickest Container solution to setup.

Very cheap price for most workloads ($0-10, or under $100 per month on average for most workloads). More on pricing here.

Windows and Linux in GA
It is not possible to vertically scale a specific ACI instance to have more CPU/Memory, you would need to redeploy that container in order to get more. You can horizontally scale by adding more containers, however, ACI does not support load balancing so you would need to manage it yourself. Azure Container Instances offers the fastest and simplest way to run a container in Azure, without having to provision any virtual machines or learning new tools—it's just your application, in a container, running in the cloud. With Azure Container Instances, you can easily run containers with a single command. Wide spectrum of scenarios including batch processing, continuous integration, and event-driven computing. We hear consistently from customers that ACI is uniquely suited to handle their burst workloads. ACI supports quick, cleanly packaged burst compute that removes the overhead of managing cluster machines. Some of our largest customers are also using ACI for data processing where source data is ingested, processed, and placed in a durable store such as Azure Blob Storage. By processing the data with ACI rather than statically provisioned virtual machines, you can achieve significant cost savings due to ACI’s granular per-second billing.

Can also be used in conjunction with AKS to elastically burst from your Azure Kubernetes Service (AKS) cluster into ACI.
App Services: Containers (PaaS) Free option available for test workloads. Moderate price for most workloads (10's to 100's per month on average for most workloads. 1000's for Isolated Service Plan Tiers). More on pricing here.

Linux fully supported in GA. Windows supported in PREVIEW mode.
Built-in auto-scaling (vertically and horizontally) and load balancing available. Just pull container images from Docker Hub or a private Azure Container Registry, and Web App for Containers will deploy the containerized app with your preferred dependencies to production in seconds. The platform automatically takes care of OS patching, capacity provisioning, and load balancing. Can be used for simple Web Apps that require scaling, do not require orchestration, and have great cost savings.
Azure Container Service (ACS) (IaaS) To be depreciated in 2020 - Do not create new applications on ACS anymore. Azure Container Service Will Retire on January 31, 2020 -- --
Azure Kubernetes Service (AKS) (PaaS) Difficult. Need to learn to use Docker and command line, Kubernetes architecture and kubectl command line, and most likely Azure command line. More on pricing here.

AKS the service has no cost. AKS master nodes also have no cost. Cost is from the VMs utilized for the worker/minion nodes. Expensive for most workloads (100's to 1000's per month on average).

Linux fully supported in GA. Windows supported in PREVIEW mode.
The cluster autoscaler (CA) (PREVIEW 01/2019) can scale your agent nodes based on pending pods. It scans the cluster periodically to check for pending pods or empty nodes and increases the size if possible. By default, the CA scans for pending pods every 10 seconds and removes a node if it's unneeded for more than 10 minutes. When used with the horizontal pod autoscaler (HPA), the HPA will update pod replicas and resources as per demand. If there aren't enough nodes or unneeded nodes following this pod scaling, the CA will respond and schedule the pods on the new set of nodes.

Currently you can setup autoscale on Kubernetes itself on AKS without worry about the PREVIEW mode cluster autoscaler (CA).
Best suited for: large enterprise micro-service architectures that need to be able to be scaled on demand quickly and need as close to 100% up-time as possible and want to be able to have rolling updates with no downtime. Usually front-facing customer applications. At the moment, best for .NET Core (cross platform) on Linux containers or other general Linux container workloads.
Reference: https://azure.microsoft.com/en-us/overview/containers/

Additional Notes and Resources
  • Container Security in Microsoft Azure - August 2018
  • AKS FAQ
  • Azure automatically applies security patches to the nodes in your AKS cluster on a nightly schedule. However, you are responsible for ensuring that nodes are rebooted as required.
  • You can connect AKS to ACI, and use Kubernetes to handle orchestration and scale.
  • Docker Swarm and DC/OS will no longer supported as orchestrators on Azure and will be depreciated in Jan 2020.