Incoming Webhook in Microsoft Teams

Microsoft Teams has been around for the last four years. Teams evolved from Skype for Business and Microsoft developed it to compete with other popular communications tools like Hangout/Slack/Zoom etc.

As all of you know we use the Microsoft Teams to talk to team or individual team members on a day to day basis. Just imagine if we can get critical information like monitoring alerts of application(s) or database(s) or any other events that is important to you or your team can be notified at real time. I think everyone will welcome an idea like this without a doubt.

As the year 2020 has been a difficult year because of COVID-19 and team members are working remotely, so I picked up this topic and did some deep dive into it, so things which are critical can be made available at one place so if we have any issue(s) it can be notified to all team members, so it becomes easy to act on it. Here I am going to walk you through the basic setup within Microsoft Teams which allows anyone to setup a webhook. Using the webhook we can send custom notifications based on requirements.

Incoming webhook are special type of Connector in Microsoft Teams that provide a simple way for an external application to share content in team channels. Microsoft Teams provides a unique URL to which you send a JSON payload with the message that you want to POST, typically in a card format. Cards are user-interface (UI) containers that contain content and actions related to a single topic and are a way to present message data in a consistent way.

Add an incoming webhook to a Microsoft Teams Channel

• Navigate to the channel where you want to add the webhook and select (•••) More Options from the shortcut navigation bar.
• Choose Connectors from the drop-down menu and search for Incoming Webhook.
• Select the Configure button, provide a name, and, optionally, upload an image avatar for your webhook.
• The dialog window will present a unique URL that will map to the channel. Make sure that you copy and save the URL — you will need to provide it to the outside service.
• Select the Done button. The webhook will be available in the team channel.

To send a message through an incoming webhook, you post a JSON payload to the webhook URL. You can also use this JSON to create cards containing rich inputs, such as text entry, multi-select, or picking a date and time. The code that generates the card and posts to the webhook URL can be running on any hosted service. These cards are defined as part of actionable messages, and are also supported in cards used in Teams bots and Messaging extensions.

Post a message to the webhook using cURL or Postman

From the command line, enter the following cURL command:

curl -H ‘Content-Type: application/json’ -d ‘{“text”: “Welcome to the world of Microsoft Teams Notification !”}’ ‘<YOUR WEBHOOK URL>’

If the POST succeeds, you should see a simple 1 output or response by curl. Check the Microsoft Teams, you should see the above message posted in your Channel.

A JSON payload to format the message with a better presentation instead of plain text message.

{

   "@type": "MessageCard",

   "@context": "http: //schema.org/extensions",

   "themeColor": "0584 ED",

   "summary": "Incoming Webhook Notification Demo",

   "sections": [{

"activityTitle": " ** Birthday of John Doe ** ",

"activitySubtitle": "May this special day be as incredible as you are.May this birthday be the start of a joyous and astonishing life ahead.Happy Birthday!",

"activityImage": "https: //pngimg.com/uploads/happy_birthday/happy_birthday_PNG7.png",

"markdown": true

}]

}

Once you’ve this basic thing setup, it opens up lot of possibility or opportunity to use it for various things like notifying your team on bugs created, pull requests or any integration with other systems to get the required information at real-time can be achieve.

Rate limiting for Connectors

Application rate limits control the traffic that a connector or an incoming webhook is allowed to generate on a channel. Teams tracks requests via a fixed-rate window and incremental counter measured in seconds. If too many requests are made, the client connection will be throttled until the window refreshes, i.e., for the duration of the fixed rate.

The limits are in place to reduce spamming a channel with a connector and ensures an optimal experience to your requirements. Please use it wisely.

Remove an incoming webhook from a Microsoft Teams Channel

• Navigate to the channel where the webhook was added and select (•••) More Options from the shortcut navigation bar.
• Choose Connectors from the drop-down menu.
• On the left, under Manage, choose Configured.
• Select the number Configured to see a list of your current connectors.
• Select Manage next to the connector that you want to delete.
• Select the Remove button and you will be presented with a Remove Configuration dialog box.
• Optionally, complete the dialog box fields and checkboxes prior to selecting the Remove button. The webhook will be deleted from the team channel.

Read More

Amazon DynamoDB

Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale. It’s a fully managed, multi-region, multi-active, durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications. DynamoDB can handle more than 10 trillion requests per day and can support peaks of more than 20 million requests per second.

This post aims to help developers and operations team to understand the strengths and weaknesses of DynamoDB.

Data Modeling

DynamoDB supports a document oriented data model. To create a table, we just define the primary / partition key. Items can be added into these tables with a dynamic set of attributes similar to MongoDB. Items in DynamoDB correspond to rows and attributes correspond to columns in RDBMS. DynamoDB supports these data types - Number, String, Binary, Boolean, Set, List, Map.

Operations Ease

As it’s a managed service from Amazon, users are abstracted away from the underlying infrastructure and interact only with the database over a remote endpoint. There is no need to worry about operational concerns such as hardware, setup/configuration, throughput capacity planning, replication, software patching, or cluster scaling — making it very easy to get started.

In fact, there is no way to access the underlying infrastructure components such as the instances or disks. DynamoDB tables require users to reserve read capacity units (RCUs) and write capacity units (WCUs) upfront. Users are charged by the hour for the throughput capacity reserved (whether or not these tables are receiving any reads or writes).

Linear Scalability

DynamoDB supports auto sharding and load-balancing. This allows applications to transparently store ever-growing amounts of data. The linear scalability of DynamoDB is good for applications that need to handle growing datasets and IOPS requirements. However, this linear scalability comes with extreme costs beyond a certain point.

Amazon Ecosystem Integration

DynamoDB is well integrated into the AWS ecosystem. It means that end users do not need to figure out how to perform various integrations by themselves. Below are couple of examples of these integrations:

• Data can easily and cost-effectively be backed up to S3
• Security and access control is integrated into AWS IAM

Cost Effectiveness

DynamoDB’s pricing model can easily make it the single most expensive AWS service for a fast growing data set. Here are some reasons:

Higher provisioning to handle partitions

In DynamoDB, the total provisioned IOPS is evenly divided across all the partitions. Therefore, it is extremely important to choose a partition key that will evenly distribute reads and writes across these partitions.

Cost explodes for fast growing data sets

As data grows, so do the number of partitions in order to automatically scale out the data (each partition is a maximum of 10GB). However, the total provisioned throughput for a table does not increase. Thus, the throughput available for each partition will constantly decrease with data growth.

Indexes will result in additional cost

Applications wanting to query data on attributes that are not a part of the primary key need to create secondary indexes. Local Secondary Indexes do not incur extra cost, but Global Secondary Indexes require additional read and write capacity provisioned leads to additional cost.

Additional cost for caching tier

Applications wanting less latency, we should add cache to increase the performance. The caching tier DAX or Elastic Cache is an additional expense on the top of the database tier.

The ideal workloads for DynamoDB should have the following characteristics:

• Low write throughput.
• Small and constant dataset size, doesn’t have unknown data growth.
• Constant or predictable read throughput, should not be explosion or unpredictable.
• Applications that can tolerate eventual consistent reads, the least expensive data access operation in DynamoDB.

Some guidelines if you are going to use DynamoDB.

• Use GUID’s or Unique Attributes, instead of incremental IDs.
• Don’t try to normalise your tables.
• Keeping pre-computed data upon updates is efficient with DynamoDB if you need to query them often.
• Don’t try to keep many relationships across tables. This will end up needing to query multiple tables to retrieve required attributes.
• Design your tables, attributes, and indexes thinking of the nature of queries.
• Think about item sizes and using indexes effectively when listing items to minimise throughput requirements.
• Avoid using DynamoDB Scan operation whenever possible.

Read More

SLF4J vs LOG4J

Image Source: https://www.educba.com/

Do I go for SLF4j or LOG4j or both? Classic question from my team.

SLF4j has been around for sometime now and has been adopted heavily across. But certain things never get over. Hence I thought of writing down my thoughts as a blog to help others.

Going back to the question; SLF4j or LOG4j ? I think this question itself incorrect. SLF4j and LOG4j focus on different areas and they are meant to do two different things. It's like comparing apples and oranges.

SLF4j is a logging facade. It doesn’t do logging by itself and depends on a logging component like LOG4j, Logback or JLogging. SLF4j is an API designed to give generic access to many logging frameworks. So your log code within the application level remains same but the underlying logging framework can be switched without any kind of actual source code changes.

Once you get used to the syntax of SLF4j, then you don’t need to worry about syntax for other logging frameworks. Another major feature of SLF4j which convinced me to use it over my long time favourite LOG4j, is presence of placeholder, which is represented as {} in code. Placeholder is pretty much same as %s in format() method of String, because it gets substituted by actual string supplied at runtime. This not only reduces the number of string concatenations in your code, but also cost of creating string objects. Since Strings are immutable and they are created in String pool, they consume heap memory and most of the time they are not needed e.g. a String used in DEBUG statement is not needed when your application is running on ERROR level in production.

By using SLF4j, you can defer String creation at the runtime, which means only required Strings will be created. If you have been using LOG4j then you must be already familiar with a workaround of putting debug statement inside if() condition, but SLF4j placeholders are much better than that.

LOG4j Style:

if (LOGGER.isDebugEnabled()) {
                LOGGER.debug("Initiating Batch Processing... RequestId: " + requestId + ", Region: " + region);
}

SLF4j Style:

LOGGER.debug("Initiating Batch Processing... RequestId: {}, Region: {}", requestId, region);

You might be thinking what if I have multiple parameters? Well you can either use variable arguments version of log methods or pass them as Object array. It is very convenient and efficient way of logging. Remember, before generating final String for logging the message, this method checks if a particular log level is enabled or not, which not only reduces memory consumption but also CPU time involved in executing those String concatenation instruction in advance. It’s also worth knowing that logging has severe impact on performance of the application, and it’s always advised to have only mandatory logging in production environment.

Code Snippet from org.slf4j.impl.Log4jLoggerAdapter:

public void debug(String format, Object arg1, Object arg2) {
        if (logger.isDebugEnabled()) {
        FormattingTuple ft = MessageFormatter.format(format, arg1, arg2);
        logger.log(FQCN, Level.DEBUG, ft.getMessage(), ft.getThrowable());
        }
  }

Summary:

• SLF4j provides place holder based logging, which improves readability of code by removing checks like isDebugEnabled(), isInfoEnabled() etc.
• By using SLF4j logging method, you defer cost of constructing logging messages (String), which is both memory and CPU efficient.
• On a side note, less number of strings means less work for Garbage Collector; in turn better throughput and performance!
• Using SLF4j in your source will make it independent of any particular logging implementation i.e., no need to manage multiple logging configurations for multiple libraries.

So essentially, SLF4j does not replace LOG4j; they work together, hand in hand. SLF4j removes the dependency on LOG4j from your application and makes it easy to replace it in future with more capable library without any kind of source code changes.

Read More

MyBatis Database Migration Tool

Evolving databases has been one of the major challenges for software development. Often regardless of our software development methodology, the database follows a different change management process. The tools of the past have been GUI centric, proprietary for a particular database and/or carried a steep license cost. Yet, at the end of the day they suffered from the same challenges.

The MyBatis Migration Maven plugin is a simple command line tool that helps us to manage the database schema changes in a more systematic way where manual work can be avoided. It can be used for any relational database systems like postgres, mysql, oracle etc.

For migration, you create the sql script files as usual which will contain one more sql statements and save the file with a specific naming convention provided by the MyBatis Miration. File naming convention YYYYMMDDHHMMSS_file_name.sql. The timestamp provided in the filename should be unique otherwise the migration will abort with an error.

• Eg:- 20170506163015_create_member_details.sql

One of the good thing about MyBatis Migration is that the script file will contain sql statements to make the changes as well as to rever those changes. Hence the script file has been divided into two sections like below

• --// First migration. -- Migration SQL that makes the change goes here. 
• --//@UNDO -- SQL to undo the change goes here.

Here are the steps to integrate your new or an existing project with MyBatis Migration.

Add Maven Plugin and Dependency

<plugin>
         <groupId>org.mybatis.maven</groupId>
         <artifactId>migrations-maven-plugin</artifactId>
         <version>1.1.2</version>
         <configuration>
             <repository>src/main/resources/db/</repository>
             <output>dist/migration-name.sql</output>
         </configuration>
             <dependencies>
                 <dependency>
                     <groupId>org.postgresql</groupId>
                     <artifactId>postgresql</artifactId>
                     <version>9.4-1205-jdbc4</version>

                 </dependency>
             </dependencies>
     </plugin>

Add a property file

Under your src/main/resources add a sub-directory named environments. And create a property file and name it say development.properties. You can also create test.properties and production.properties files. The environment can be specified when running a migration by using the –env=<environment> option (without the path or “.properties” part). Below is the template which should be used for the property file.

## Base time zone to ensure times are consistent across machines
time_zone=GMT+0:00

## The character set that scripts are encoded with
# script_char_set=UTF-8

## JDBC connection properties.
driver=
url=
username=
password=

# Name of the table that tracks changes to the database
changelog=db_changelog

# If set to true, each statement is isolated in its own transaction.
# Otherwise the entire script is executed in one transaction.
auto_commit=false

# This controls how statements are delimited. 
# By default statements are delimited by an end of line semicolon. 
# Some databases may (e.g. MS SQL Server) may require a full line delimiter such as GO.
delimiter=;
full_line_delimiter=false

# This ignores the line delimiters and simply sends the entire script at once.
# Use with JDBC drivers that can accept large blocks of delimited text at once.
send_full_script=false

# Custom driver path to avoid copying your drivers
# driver_path=

Maven Goals

mvn migration:status

As the goal is straight forward, it provides the status of the migration whether the script has been executed or not. It provides a simple tabular information that contains id (timestamp provided in the filename), description (filename after the timestamp) and applied_at specifies when the migration has run for each file on the database.

mvn migration:up

A goal which will execute all pending migation scripts one by one based on the timestamp provided in the order.

mvn migration:down

A goal which will execute the undo section of the last migrated script.

mvn migration:pending

A goal which will execute all pending migation scripts one by one based on the timestamp provided in the order.

An alternate for MyBatis Migration is FlyWay and Liquidbase. You can download the fully functional working copy of the project from GitHub.

In case if you are wondering if we have any similar tool for MongoDB, then you can explore MongoBee which works similarly but only thing it lacks is reverting the changes as it’s not supported currently. To know more about it, please visit MongoBee.

Read More

Java String Concatenation

Image Source: https://www.educba.com/

Have you been told many times, don’t use + operator to concatenate Strings? We know that it is not good for performance. How do you really know whether is it true or not? Do you know what is happening behind the hood? Why don’t we go ahead and explore all about String concatenation?

In the initial versions of java around JDK 1.2 every body used + to concatenate two String literals. Strings are immutable, i.e., a String cannot be modified. Then what happens when we write the following code snippet.

• String message = "WE INNOVATE ";
• message = message + "DIGITAL";

In the above java code snippet for String concatenation, it looks like the String is modified but in reality it is not. Until JDK 1.4 the StringBuffer was used internally for concatenation and from JDK 1.5 StringBuilder is used to concatenate. After concatenation the resultant StringBuffer or StringBuilder is changed to String object.

You would have heard from java experts that, “don’t use + operator but use StringBuffer”. If + is going to use StringBuffer internally, what big difference it is going to make in String concatenation using + operator?

Look at the following examples. I have used both + and StringBuffer as two different cases.

• Case 01, I am just using + operator to concatenate.
• Case 02, I am changing the String to StringBuffer and then doing the concatenation. Then finally changing it back to String.

I have used a timer to record the time taken for an example of String concatenation.

public class StringConcatenateExample {
   private static final int LOOP_COUNT = 50000;
  
   public static void main(final String args[]) {
      long startTime, endTime;
    
      startTime = System.currentTimeMillis();
      String message = "*";
      for(int i=1; i<=LOOP_COUNT; i++) {
         message = message + "*";
      }
      endTime = System.currentTimeMillis() - startTime;
      System.out.println("Time taken to concatenate using + operator: " + endTime + " ms.");

      startTime = System.currentTimeMillis();
      StringBuilder sBuilder = new StringBuilder("*");
      for(int i=1; i<=LOOP_COUNT; i++) {
         sBuilder.append("*");
      }
      sBuilder.toString();
      endTime = System.currentTimeMillis() - startTime;
      System.out.println("Time taken to concatenate using StringBuilder: " + endTime + " ms.");
   }
  
}

Look at the output (if you run this java program the result numbers might slightly vary based on your hardware/software configuration). The difference between the two cases is extremely surprising.

You might argue, if + operator is using StringBuffer internally for concatenation, then why is this huge difference in time? Let me explain, when a + operator is used for concatenation, see how many steps are involved behind the scenes:

• A StringBuffer object is created.
• Message is copied to the newly created StringBuffer object.
• The “*” is appended to the StringBuffer (concatenation).
• The result is converted back to a String object.
• The message reference is made to point at that new String.
• The old String that message previously referenced is then made null.

It is now clear that there are serious performance issues that can result if you use + operator for concatenation and why is it so important to use StringBuffer or StringBuilder (from java 1.5) to concatenate Strings.

And on a side note, the StringBuffer is slower compared to StringBuilder because it’s a thread safe object, where all the methods are synchronised, so you need to take a decision wisely on usage based on your requirement.

Read More