Azure VPN Gateway with Cisco ASA using Routing

The Azure VPN Gateway and Cisco ASA can encounter routing-type issues when configured together. To resolve this, the UsePolicyBasedTrafficSelectors must be enabled in the Azure Connection. The provided code is a PowerShell script that retrieves the specified Azure virtual network gateway connection and creates a new IPsec policy with specific parameters. The script then sets the IPsec policies for the connection to the new policy and enables UsePolicyBasedTrafficSelectors to solve the routing issue.

$RG1 = "****************"

This line declares a variable $RG1 and sets its value to a string of asterisks. This is likely just a placeholder for the actual resource group name.

$Connection16 = "****************"

Similar to the first line, this line declares a variable $Connection16 and sets its value to a string of asterisks. This is likely just a placeholder for the actual connection name.

$connection6 = Get-AzureRmVirtualNetworkGatewayConnection -Name $Connection16 -ResourceGroupName $RG1

This line retrieves the virtual network gateway connection object for a connection with the specified name ($Connection16) in the specified resource group ($RG1). The connection object is assigned to the variable $connection6.

$newpolicy6 = New-AzureRmIpsecPolicy -IkeEncryption AES256 -IkeIntegrity SHA384 -DhGroup DHGroup24 -IpsecEncryption AES256 -IpsecIntegrity SHA1 -PfsGroup PFS24 -SALifeTimeSeconds 28800 -SADataSizeKilobytes 4608000

This line creates a new IPsec policy object ($newpolicy6) with the specified settings for IKE encryption, integrity, DH group, IPsec encryption, integrity, Perfect Forward Secrecy (PFS) group, Security Association (SA) lifetime, and SA data size.

Set-AzureRmVirtualNetworkGatewayConnection -VirtualNetworkGatewayConnection $connection6 -IpsecPolicies $newpolicy6

This line updates the virtual network gateway connection object ($connection6) with the new IPsec policy ($newpolicy6) created in the previous step.

Set-AzureRmVirtualNetworkGatewayConnection -VirtualNetworkGatewayConnection $connection6 -IpsecPolicies $newpolicy6 -UsePolicyBasedTrafficSelectors $True

This line updates the virtual network gateway connection object ($connection6) again, this time enabling policy-based traffic selectors by setting the -UsePolicyBasedTrafficSelectors parameter to $True. This is necessary to resolve routing issues that can occur when configuring the Azure VPN Gateway with a Cisco ASA.

PS Azure:\> $connection6.UsePolicyBasedTrafficSelectors

True

Azure:\

PS Azure:\> $connection6.IpsecPolicies

Docker Management using Portainer

Portainer is a lightweight management UI that allows easy management of Docker environments, including creating, deploying, and managing containers, services, and stacks. It is particularly useful for those who are new to Docker or those who prefer a visual interface over command-line management.

To install Portainer with a persistent container, you can follow these steps:

• Pull the Portainer image: docker pull portainer/portainer
• Create a directory for Portainer data: mkdir -p /mnt/docker/portainer/data
• Create a Docker service for Portainer with the following command:  

docker service create \ --name portainer \ --publish 9090:9000 \ --constraint 'node.role == manager' \ --mount type=bind,src=/mnt/docker/portainer/data,dst=/data \ --mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \ portainer/portainer \ -H unix:///var/run/docker.sock

 

the above command will create a new Docker service named "portainer" with a published port of 9090, mounted volume for persistent data, and a constraint for the node role of "manager".

• Access the Portainer UI by visiting the IP address or hostname of the Docker swarm manager node on port 9090 in a web browser.
• Create a new user account and start managing your Docker environment using the Portainer UI.

Access the Portainer UI by visiting the IP address or hostname of the Docker swarm manager node on port 9090 in a web browser.

Qubole load CSV with spark

This is a code snippet using Spark on Qubole to load a CSV file into a DataFrame, register it as a temporary table, and create a permanent table from the data in the temporary table.

The first line of code reads the CSV file from an S3 location into a DataFrame. The options set for the format, delimiter, header, and inferSchema specify how the CSV file should be read and parsed.

val df = sqlContext.read.format("com.databricks.spark.csv")
                    .option("delimiter", "|")
                    .option("header", "true")
                    .option("inferSchema", "true")
                    .load("s3://*****.CSV")

The second line of code registers the DataFrame as a temporary table, which can be used for querying.

df.registerTempTable("temp-table")

The third line of code creates a permanent table in a specified database by executing an SQL query on the temporary table. The query selects all the columns and rows from the temporary table and creates a new table with the same data in the specified database.

sqlContext.sql("""
create table database.table as
select * from temp-table
""")