{"componentChunkName":"component---src-pages-author-author-yaml-id-js","path":"/author/chris-yee/","result":{"data":{"allMarkdownRemark":{"edges":[{"node":{"id":"ac280a24-9415-5830-99fb-1ea29922fdc0","html":"

Web authentication or WebAuthn is a new standard for authentication published by the World Wide Web Consortium and supported by the FIDO alliance. The standard works by providing a way for users to authenticate through a third-party authenticator. These authenticators can be built into the operating system software, like Windows Hello or Android biometrics, or through external authenticators, like a USB authenticator.

\n

This article will cover a high-level description of how WebAuthn works and two different ways an application can be set up to use the WebAuthn standard to authenticate users. One is through a basic registration flow where the authenticator data is stored upon registration and must be used to log in. The other is a flow where the authenticator can be used as a second-factor authentication method, prompted after a user logs in. For more information on this, resources like the World Wide Web Consortium documentation can be useful.

\n

WebAuthn Introduction

\n

The WebAuthn authentication process involves the interaction between three entities: The relying party, which is the server that handles the credentials of the authenticator and verifies it, a web browser that should support the web authentication protocol, and the authenticator itself.

\n

The registration process begins when the client generates some sort of identifying information about themselves. Typically, this would be a user name or an email from the user but could be another identifier sent from the client, such as a randomly generated GUID. Using the Web Authentication APIs with JavaScript, the client can request the relying party to register the authenticator credentials.

\n

\n \n

  • Resident Key Required Boolean - By default, this value is set to false, which means that private key and authentication credentials are stored on the relying party server. However, if this option is set to true, the authenticator will store the private key credentials instead. With certain flows, by setting the resident key to required, only the authenticator's ids will be stored on the relying party server. This would allow for authentication systems to validate a user via the authenticator and identify them.
    • \n
  • User Verification - This field can hold three values: Preferred by default, Discouraged, or Required. User verification is when the user can be identified to be one that is attempting the authentication request. If an authenticator supports user verification, the authenticator must set a flag during validation and sent to the relying party. When the authenticator attempts verification, it can be done through different means, such as password, PIN, or biometrics.
    • \n
  • Attestation - Attestation certificates have the option to be created through this option. As a result, the certificate will contain details about the hardware being used as an authenticator and ensure that the device used for authentication is trustworthy. If the authentication flow does not require this, you can set it to None to reduce the workload done by the relying party to validate the attestation certificate. By setting the option to Indirect, the relying party allows the attestation certificate to be generated by a third party. This could allow the relying party to receive less private information surrounding the actual authenticator and use only what is needed. Lastly, if set to direct, the authenticator itself must generate the attestation certificate to be verified.
    • \n\n

    The authentication or login process is similar to the registration process. Using the JavaScript libraries, the authenticator passes identifying information to the relying party. The relying party returns a challenge, which should be validated by the user, processed by the authenticator, and finally validated by the relying party. User verification is checked from the options, but a majority of the options have been reduced as no credentials are generated or saved on either end.

    \n

    Setting up WebAuthn for your Application

    \n

    When setting up WebAuthn as an authentication method for your application, two considerations are to use the standard as a primary registration and authentication method or as a second-factor authentication method.

    \n

    As a primary authentication method, the authenticator identification and credentials are stored by the server at the same time the user account is created.

    \n

    To begin, use the Web Authentication API to initiate the flow:

    \n
    Let publicKeyCredentialCreationOptions = {\n    user: {\n        id: “1234”,\n        name: "example@example.com",\n        displayName: "example",\n    },\n    authenticatorSelection: {\n    authenticatorAttachment: "platform",\n    },\n    attestation: "direct"\n};\n\nconst credential = await navigator.credentials.create({\n    publicKey: publicKeyCredentialCreationOptions\n});
    \n

    For this section, the creation options are generated by the client. In other scenarios, including this blog's example, the options are generated from the relying party by retrieving them via an API call.

    \n

    After the browser triggers this code, the authenticator will be prompted for validation. Since the authenticatorAttachment is set to 'platform', it would use the device's authenticator, like Windows Hello or an Android fingerprint scan.

    \n

    The data returned from the creation of the credentials will have a structure like this:

    \n
    PublicKeyCredential {\n    id: 'ADSUllKQmbqdGtpu4sjseh4cg2TxSvrbcHDTBsv4NSSX9...',\n    rawId: ArrayBuffer(59),\n    response: AuthenticatorAttestationResponse {\n        clientDataJSON: ArrayBuffer(121),\n        attestationObject: ArrayBuffer(306),\n    },\n    type: 'public-key'\n}
    \n

    This object should be passed to the relying party to complete the registration. The relying party verifies that the credentials passed in are correct and stores the identifier and credentials in the data store. In addition to storing the credentials object, any other registration variables should be passed in this step to register on your data store.

    \n

    To use this flow as part of a second-factor authentication process, the user account should be created before initiating this process with an already existing authentication method, like a password system. After the user account is created, running this flow should update the current user in the data store with the credential data. This will allow the second-factor authentication credentials to be configured and triggered after a password login.

    \n

    Most programming languages have a library built that supports WebAuthn for a relying party server. For example, Duo has created them for Python and Go. Using one of these libraries during the development of the relying party server will simplify the process. These resources also provide demos on how to implement their libraries to handle the credential data generated by the JavaScript Web Authentication APIs.

    \n

    Conclusion

    \n

    Although the usage of WebAuthn is not widespread at the moment, the potential to authenticate a user passwordless or with the extra security of an authenticator allows it to be a strong contender in the future over regular password authentication flows. Hopefully, this blog helps you get started on your authentication apps with WebAuthn.

    \n","frontmatter":{"title":"WebAuthn: A Guide To Authenticate Your Application","author":{"id":"Chris Yee","github":null,"avatar":null},"date":"December 09, 2020","updated_date":null,"tags":["WebAuthn","FIDO","Authentication"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/07ab3c5a163821d532aedbc23c7a48be/ee604/webauthn-logo.png","srcSet":"/static/07ab3c5a163821d532aedbc23c7a48be/69585/webauthn-logo.png 200w,\n/static/07ab3c5a163821d532aedbc23c7a48be/497c6/webauthn-logo.png 400w,\n/static/07ab3c5a163821d532aedbc23c7a48be/ee604/webauthn-logo.png 800w,\n/static/07ab3c5a163821d532aedbc23c7a48be/f3583/webauthn-logo.png 1200w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Chris Yee","slug":"/engineering/webauthn-authentication-application/"}}},{"node":{"id":"51e17045-40a3-54bd-a1d8-28b66237ca68","html":"

    Introduction

    \n

    Redis's usage as both a temporary data store and even as a permanent data store is increasing today, as memory has started becoming much cheaper. Redis' quick memory-based CRUD operations allow database interactions to happen significantly faster than interactions with other databases that use drive-based storage. This blog is about three different methods to interact with your Redis data.

    \n

    This blog will require having access to a Redis instance. Guides can be found to set up a local Redis instance for Windows here and for UNIX based machines like Mac or other Linux distros, it can be found here. Alternatively, you can use a remote instance, but be aware that some of the commands in this blog may delete or modify the data.

    \n

    Redis Insight

    \n

    For users that do not like working with command-line interfaces, the first tool introduced in this blog is a GUI interface called RedisInsight. This can be found here. After filling in their download form and installing, running the application will start a server on the localhost, which you can access with a browser.

    \n

    After installing and running the application, you will add the connection information for your Redis instance. If running a local instance of Redis, the connection information in the image will typically suffice. If connecting to a remote instance, you might need to fill in some additional details for TLS, such as connection certificates.

    \n

    \n \n \n

    \n

    When connected to your Redis instance, the dashboard UI appears. Clicking the Browser sidebar tab will display all existing keys in your database.

    \n

    \n \n \n

    \n

    From here, CRUD operations are relatively straightforward. Key-value pair creation can be done by using the ADD KEY button. This will prompt for the type of key-value pair to be added (Hash, String, JSON, etc.) and the data that will be stored with the key.

    \n

    If a specific key is required to be queried, fill in the field next to the blue filter icon with search criteria, then click the icon. It will prompt the type of data stored with the key, and the results will be displayed.

    \n

    \n \n \n

    \n

    Finally, keys can be deleted by clicking the trash can icon on the right side.

    \n

    Redis CLI

    \n

    The Redis command-line interface is a more common approach for developers to interact with data from a Redis store. If you followed the steps at the beginning of the blog, the CLI should already be installed. To do a ping test to the server, type the following command in the terminal:

    \n
    chris@LR-ENG-17:~$ redis-cli -h localhost -p 6379 ping\nPONG
    \n

    And to connect, run the previous command without pinging:

    \n
    chris@LR-ENG-17:~$ redis-cli -h localhost -p 6379\nlocalhost:6379>
    \n

    The basic command to get a key-value pair is to call get with the key:

    \n
    localhost:6379> get key:1\n"value1"
    \n

    Setting a value can be done with the set command:

    \n
    localhost:6379> set key:1 value1new\nOK\nlocalhost:6379> get key:1\n"value1new"
    \n

    Creating a value can also be done in the same way:

    \n
    localhost:6379> get key:3\n(nil)\nlocalhost:6379> set key:3 value3new\nOK\nlocalhost:6379> get key:3\n"value3new"
    \n

    And deletion can be done with the del command:

    \n
    localhost:6379> del key:3\n(integer) 1\nlocalhost:6379> get key:3\n(nil)\nlocalhost:6379>
    \n

    This is only the simple basics of the Redis CLI. You can do more advanced operations through the CLI, such as the flushall command, which wipes your database. It’s a powerful tool, so be careful when experimenting with it. An extensive list of commands can be found here.

    \n

    With Your Chosen Programming Language

    \n

    With Redis’ widespread usage, almost all major industry programming languages have either community or official supported releases of libraries that can be used to interact with your database. In this section, I will be using Node.JS, and one of the support Redis libraries called node-redis.

    \n

    Initialization of the package in the code can be done by calling the following command after initializing Node in your directory:

    \n
    npm install redis
    \n

    When the process is finished, the redis module should be imported within your code, where you can set up the Redis connection and any options that need to be included:

    \n
    const redis = require("redis")\n\nconst redisOption = {\n  host: "127.0.0.1",\n  port: "6379",\n}\n\nconst client = redis.createClient()
    \n

    For an extensive list of options supported by the library for connection, check this link.

    \n

    After connecting, calling the on method on the client will allow the connection to open and run commands.

    \n

    The commands used by the library are a 1 to 1 mapping of the commands used by the Redis CLI, so they follow the same method calls. To update or create an entry in Redis, the most basic method is to use the set command:

    \n
    client.set("key:3", "value3new")
    \n

    Getting a value can be done with the get method:

    \n
    client.get("key:3")
    \n

    Deleting a value can be done with the del method:

    \n
    client.del("key:3")
    \n

    The methods used by the client also support callbacks. For example, if the operations should be console logged, we can include a callback to redis.print after each method:

    \n
    client.set("key:3", "value3new", redis.print)\nclient.get("key:3", redis.print)\nclient.del("key:3", redis.print)
    \n

    The output of this in the console would be:

    \n
    Reply: OK\nReply: value3new\nReply: 1
    \n

    Conclusion

    \n

    This blog has only touched the surface of the different operations Redis can do. There are many more resources online that can help you go further into your understanding of Redis. Whether using Redis as a temporary store, like a cache, or using it as a primary, permanent store, the operations' speed is something to be acknowledged and considered when building your next big app.

    \n","frontmatter":{"title":"Three Ways to do CRUD Operations On Redis","author":{"id":"Chris Yee","github":null,"avatar":null},"date":"October 21, 2020","updated_date":null,"tags":["Redis","NodeJs","CLI","GUI"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/4eda1ce5a0f541d97fdf27cd88bf2a49/ee604/index.png","srcSet":"/static/4eda1ce5a0f541d97fdf27cd88bf2a49/69585/index.png 200w,\n/static/4eda1ce5a0f541d97fdf27cd88bf2a49/497c6/index.png 400w,\n/static/4eda1ce5a0f541d97fdf27cd88bf2a49/ee604/index.png 800w,\n/static/4eda1ce5a0f541d97fdf27cd88bf2a49/f3583/index.png 1200w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Chris Yee","slug":"/engineering/three-ways-to-do-crud-operations-on-redis/"}}},{"node":{"id":"81cb1214-08b9-55bf-9b37-848ad8a629a7","html":"

    Golang Maps is a compilation of unordered key-value pairs. It is commonly used because it offers simple searches and values with the aid of keys that can be retrieved, modified or deleted. It is a reference to a table with a hash.

    \n

    This blog will cover the basic use of maps in Go, and how a newcomer to the language may utilize them for their applications.

    \n

    What is a Map?

    \n

    Maps (also called dictionaries) are a very useful tool in helping to store and organize objects to be accessed in an efficient method.

    \n

    Most basic implementations of a map involve using a key to access a value in the map, resulting in key-value pairs, in which one key is associated with a specific value in the map. Within Golang, maps follow this definition.

    \n

    \n \n \n

    \n

    How to create a map?

    \n

    Initialization of a map can be done using the make command. This is similar to the initialization of a slice:

    \n
    mapObject := make(map[string]string)
    \n

    In this case, mapObject is a map that uses strings as a key to map to another string. When creating a map, the key type must be a type that is Comparable, or more specifically types that can be compared using the == operator.

    \n

    Examples of valid key types include booleans, numbers, strings and several other primitives can be used as keys.

    \n

    One more thing to note is that structs can be used as a key, provided that all the properties of the struct are Comparable.

    \n

    Maps can also be created using a map literal. For an empty map:

    \n
    mapObject := map[string]string{}
    \n

    Or for a map with initial data:

    \n
    mapObject := map[string]string{\n\t“Key1”: “Value1”,\n\t“Key2”: “Value2”,\n\t“Key3”: “Value3”,\n}
    \n

    How to use Go Maps (with Examples)

    \n

    Setting

    \n

    Interaction with Go maps are similar to the dictionaries and maps of other languages. A simple way to return a value associated with the key is to use bracket notation. For example, to set a string value mapped to a string key:

    \n
    mapObject[“test”] = “test_value”
    \n

    Fetching

    \n

    \n \n \n

    \n

    Retrieving a value uses the same format.

    \n
    result := mapObject[“test”]
    \n

    The value of result will be the value assigned to “test” in the map. In the case where no key is found, the zero value of the type is returned instead. In this case, if there is no associated value with “test”, an empty string is returned.

    \n

    A boolean that returns the key’s existence can also be returned if two arguments are assigned from the retrieval. For example:

    \n
    result, exists := mapObject[“test”]
    \n

    If “test” exists in the map, result will be the value associated with “test” and exists will be true. If “test” does not exist, result will be an empty string and exists will be false. This is useful when the map being used contains zero-values to distinguish between existence or whether the value is just zero.

    \n

    Deleting

    \n

    Using Go’s built in delete method, key-value pairs can be deleted off the map. To delete the previous “test” key from the map, the following can be called:

    \n
    delete(mapObject, "test")
    \n

    The delete method does not have a return value, so if the key does not exist in the map, nothing will happen.\nLength

    \n

    The number of key-value pairs of a map can be found using the len method:

    \n
    length := len(mapObject)
    \n

    Iterating

    \n

    Using Go’s range keyword, a map can be iterated through.

    \n
    for key, value := range m {\n    fmt.Println("Key:", key, "Value:", value)\n}
    \n

    There is no specific order in which the map is iterated. If a specific order is needed, a slice or other data structure can be used to store or hold data which can then be sorted and iterated through.

    \n

    Concurrency

    \n

    Although Go has a lot of support for concurrency through the use of goroutines and channels, maps alone are not a reliable approach to handling data in a concurrent setting.

    \n

    To work with maps that support concurrency, a separate synchronization method should be used, like sync.RWMutex. Alternatively, an open source map package which implements the synchronizations can be used.

    \n

    Conclusion

    \n

    With the market space and number of applications being created by Go increasing, hopefully this blog will help touch onto the basics of one of the major structures in Go.

    \n

    Although it doesn’t support concurrency, maps in Go are still a useful tool in most applications developed in Go, to reliably access and manage data.

    \n

    If you want to learn more about golang here is a quick guide on, how to send email with golang. If you like what you read leave a comment or any question and will communicate further.

    \n","frontmatter":{"title":"Golang Maps - A Beginner’s Guide","author":{"id":"Chris Yee","github":null,"avatar":null},"date":"September 25, 2020","updated_date":null,"tags":["Go","dictionaries","maps"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/30bb9b9901e76f3498d68913a0675ea9/ee604/index.png","srcSet":"/static/30bb9b9901e76f3498d68913a0675ea9/69585/index.png 200w,\n/static/30bb9b9901e76f3498d68913a0675ea9/497c6/index.png 400w,\n/static/30bb9b9901e76f3498d68913a0675ea9/ee604/index.png 800w,\n/static/30bb9b9901e76f3498d68913a0675ea9/f3583/index.png 1200w,\n/static/30bb9b9901e76f3498d68913a0675ea9/0dadc/index.png 1500w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Chris Yee","slug":"/engineering/golang-maps/"}}},{"node":{"id":"4a330e44-55e2-5948-91b1-7df02fdf2b2a","html":"

    The official supported drivers for Mongo for Go have been officially released for several months now and is slowly seeing more usage. Although many of the features of standard Mongo drivers have been implemented within the supported driver, the documentation for a majority of the features are still in the process of being created. This post will provide some information on how to set up custom JSON encoders so you can change how BSON operators are processed whenever Mongo documents are marshalled into JSON. For this blog, a custom date and object id encoder will be created to handle the BSON operators and transform them into more easily processed JSON.

    \n

    This blog assumes that you have a basic understanding of Go and how to write some code in the language. If you want to follow along with the guide, you will also require a working Mongo instance, as well as your environment set up for Go.

    \n

    Before interacting with the database, we can set up a quick Go application that finds data from a Mongo database and outputs it. For this example, the Mongo database is set up with a collection containing documents with a name, a date and the object id. For our blog, the Mongo document that is being worked with is quite simple:

    \n
    {\n    "_id" : ObjectId("5de6b0850f3c7b5cce642529"),\n    "Name" : "dateOne",\n    "Date" : ISODate("2019-04-08T11:55:02.658Z")\n}
    \n

    A simple main function that retrieves data from Mongo and outputs the JSON string is as follows:

    \n
    func main() {\n\tvar result struct {\n\t\tID   primitive.ObjectID `bson:"_id"`\n\t\tName string             `bson:"Name"`\n\t\tDate primitive.DateTime `bson:"Date"`\n\t}\n\n\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n\tclient, connErr := mongo.Connect(ctx, options.Client().ApplyURI(""))\n\n\tif connErr != nil {\n\t\tlog.Fatal(connErr)\n\t}\n\n\tcollection := client.Database("test").Collection("dates")\n\n\tfilter := bson.M{"Name": "dateOne"}\n\tfindErr := collection.FindOne(ctx, filter).Decode(&result)\n\tif findErr != nil {\n\t\tlog.Fatal(findErr)\n\t}\n\n\tfmt.Print(result.Date)\n\n\tdata, writeErr := bson.MarshalExtJSON(result, false, false)\n\tif writeErr != nil {\n\t\tlog.Fatal(writeErr)\n\t}\n\n\tfmt.Print(string(data))\n\tcancel()\n}
    \n

    The function MarshalExtJSON is a part of the bson package that is a part of the driver. It uses a default registry to define the rules when converting Mongo primitive types into JSON. The arguments it takes are the Go struct being passed in, whether the result should be returned in canonical form (For more details, check this link, and whether HTML strings are escaped.

    \n

    The output of this function when reading our document produces a JSON string with all the BSON operators included.

    \n
    {\n\t"_id": {\n\t\t"$oid":"5de6b0850f3c7b5cce642529"\n\t},\n\t"Name":"dateOne",\n\t"Date":{\n\t\t"$date":"2019-04-08T11:55:02.658Z"\n\t}\n}
    \n

    Using custom encoders, we can change how the bson package marshals the JSON and remove all the BSON operators. The first thing to do is to define package level variables that will hold the types of the Mongo primitives that need to be changed in the JSON response. Outside the main function, include these two lines:

    \n
    var tOID = reflect.TypeOf(primitive.ObjectID{})\nvar tDateTime = reflect.TypeOf(primitive.DateTime(0))
    \n

    Next, we can make new functions to customize how these types are handled. For the date encoding, we want to have the Date key matched with the returned date string.

    \n
    func dateTimeEncodeValue(ec bsoncodec.EncodeContext, vw bsonrw.ValueWriter, val reflect.Value) error {\n\tconst jDateFormat = "2006-01-02T15:04:05.999Z"\n\tif !val.IsValid() || val.Type() != tDateTime {\n\t\treturn bsoncodec.ValueEncoderError{Name: "DateTimeEncodeValue", Types: []reflect.Type{tDateTime}, Received: val}\n\t}\n\n\tints := val.Int()\n\tt := time.Unix(0, ints*1000000).UTC()\n\n\treturn vw.WriteString(t.Format(jdateFormat))\n}
    \n

    The parameters in this function are pre-defined within the driver package of the driver. This involves passing in the BSON encoding context, the value writer and the value that is being processed. The value writer and the value are the parameters that we use within our function.

    \n

    First, we check if the value passed in is the correct type. If there is an issue with the value, then an error is thrown during the encoding. Next, we can set the format we want our date string to appear in, and process our value into a Go Time object. After that, we can use the value writer to write the formatted date into our JSON string.

    \n

    Similar to the custom date encoding, the object id encoding function has the same structure.

    \n
    func objectIDEncodeValue(ec bsoncodec.EncodeContext, vw bsonrw.ValueWriter, val reflect.Value) error {\n\tif !val.IsValid() || val.Type() != tOID {\n\t\treturn bsoncodec.ValueEncoderError{Name: "ObjectIDEncodeValue", Types: []reflect.Type{tOID}, Received: val}\n\t}\n\ts := val.Interface().(primitive.ObjectID).Hex()\n\treturn vw.WriteString(s)\n}
    \n

    First, we check if the type is valid, and if it is, we simply write out the object id as a hex string. To create the hex string, we first convert the value into an interface, then cast it as an object id to allow the Hex function to be called.

    \n

    Now that our two functions are created to handle the BSON types, we can create our custom registry.

    \n
    func createCustomRegistry() *bsoncodec.RegistryBuilder {\n\tvar primitiveCodecs bson.PrimitiveCodecs\n\trb := bsoncodec.NewRegistryBuilder()\n        bsoncodec.DefaultValueEncoders{}.RegisterDefaultEncoders(rb)\n        bsoncodec.DefaultValueDecoders{}.RegisterDefaultDecoders(rb)\n\trb.RegisterEncoder(tDateTime, bsoncodec.ValueEncoderFunc(dateTimeEncodeValue))\n\trb.RegisterEncoder(tOID, bsoncodec.ValueEncoderFunc(objectIDEncodeValue))\n\tprimitiveCodecs.RegisterPrimitiveCodecs(rb)\n\treturn rb\n}
    \n

    In this function, we are setting up our registry to use our custom encoders, while also setting up default encoders for all other types that are being marshalled. Using the bsoncodec package, the NewRegistryBuilder function is called to initialize a new registry. Next, we set up this registry with the default encoders and decoders that are pre-programmed in the driver using RegisterDefaultEncoders. During this step, we can also register encoders and decoders for raw types by calling RegisterPrimitiveCodecs. Finally, we can include our custom encoders by calling RegisterEncoder on the custom registry struct. Our encoder will be called whenever the type defined in the first argument in RegisterEncoder is found in the struct being marshalled.

    \n

    To ensure our new custom registry is being used, we have to do two things. The first is to build the custom registry. This should be done before the marshalling of the struct.

    \n
    var customRegistry = createCustomRegistry().Build()
    \n

    Next, we need to change the marshalling method to use the new custom registry instead of the default one. We can do this by changing the method being used to marshal the struct.

    \n
    customData, writeErr := bson.MarshalExtJSONWithRegistry(customRegistry, result, false, false)
    \n

    The function MarshalExtJSONWithRegistry takes the same parameters as MarshalExtJSON, but also requires a custom registry to be passed in as the first parameter. We pass in the custom registry we created into the method to allow the marshalling to use our custom encoders. As a result, when printing our new JSON string output, we get this:

    \n
    {"_id":"5de6b0850f3c7b5cce642529","Name":"dateOne","Date":"2019-04-08T11:55:02.658Z"}
    \n

    The Mongo driver for Go is still in active development and new versions continue to be released. Hopefully this blog provides some useful information on how the driver works while new documentation is being created.

    \n","frontmatter":{"title":"Custom Encoders in the Mongo Go Driver","author":{"id":"Chris Yee","github":null,"avatar":null},"date":"December 03, 2019","updated_date":null,"tags":["Go","Golang","MongoDriver"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.25,"src":"/static/0bf88d6b53aaaf119a2556b41f4c8383/ee604/mongo-drivers.png","srcSet":"/static/0bf88d6b53aaaf119a2556b41f4c8383/69585/mongo-drivers.png 200w,\n/static/0bf88d6b53aaaf119a2556b41f4c8383/497c6/mongo-drivers.png 400w,\n/static/0bf88d6b53aaaf119a2556b41f4c8383/ee604/mongo-drivers.png 800w,\n/static/0bf88d6b53aaaf119a2556b41f4c8383/30c15/mongo-drivers.png 936w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Chris Yee","slug":"/engineering/custom-encoders-in-the-mongo-go-driver/"}}},{"node":{"id":"64cdfe0e-5159-596b-944f-78d57d35e11b","html":"

    Bots are tools that are created to automate tedious processes and reduce work. For example, chatbots automate replies to users for customer support, and search bots are used to populate search results on a Google search. However, there are many bots that are crafted for the malicious self-interest of a party. Examples of these hostile bots include DDOS (Direct Denial of Service) botnets and spam bots. This post will provide some information on how to implement bot protection to protect your systems from these nasty bot attacks.

    \n

    CAPTCHA:

    \n

    One of the most popular methods of bot protection that is used today is CAPTCHA, which is provided through companies such as ReCAPTCHA, NuCaptcha and Solve Media. CAPTCHA, which stands for “Completely Automated Public Turing Test to tell Computers and Humans Apart”, is an anti-bot measure which consists of a challenge which a user must complete to verify if the user is human. Examples of challenges include translating images of distorted text, or recognition of objects in an image which match a given word. CAPTCHAs are useful in blocking automated form submissions by bots and are constantly being updated to be more friendly to human users. Some of the latest CAPTCHA implementations only require the user to click on a checkbox to pass their validation check.

    \n

    An example of a ReCAPTCHA with distorted text

    \n

    \n

    To implement CAPTCHA using Google’s ReCAPTCHA solution, you can access Google reCAPTCHA bot protection and login with your Google account. Following that you will be redirected to an interface where you can register your site. Different types of CAPTCHAs can be set up for different events on your domain and can be built to match your use case.

    \n

    An example of a ReCAPTCHA with a checkbox validation

    \n

    \n

    CAPTCHAs have a variety of uses, and can be used to prevent automated form completions and even prevent access to your domain. Setting up a CAPTCHA detection solution for different scenarios will typically provide a strong bot defense for your system.

    \n

    Spam Honeypots:

    \n

    Spam honeypots are traps that ensnare bots by placing hidden input fields within a form that reject registration upon being filled in. Bots that use detect form fields through HTML may be programmed to fill in all the input fields in a form including the honeypot. Meanwhile, since the fields are hidden, human users should not be able to see the honeypot and should not be filling in the field.

    \n

    The implementation of a honey pot can be as simple as implementing an extra form field onto your page that should not be filled in. Hide the element using CSS and set up logic to prevent users that fill in the field from successfully completing the form. A simple implementation can be done with this code:

    \n
    <input id="first-name-input" type="text" name="firstname" value="Fill me in"> <input style="display: none;" id="honeypot-input" type="text" name="honeypot"> <button type="button" onclick="submitForm()" value="button">Submit</button> <button type="button" onclick="fill()" value="button">Fill Honeypot</button> <script>    let submitForm = function() {         let honeypot = document.getElementById("honeypot-input").value;         if(!honeypot) { \n   // Handle input submit            \n        console.log(“Pass”);         } \n   else { \n   // Handle honeypot error            \n    console.log(“Fail”);  }  } \n     let fill = function() {  document.getElementById("honeypot-input").value = "Example";     } \n</script>
    \n

    With the implementation of the above code, if a bot is setup to fill in all input fields on your web page, then the hidden honeypot input will be filled in and the bot will be detected. On the other hand, if a normal user attempts to complete the form on the page, the honeypot would be invisible and the registration will be successful.

    \n

    Honeypots are a solution to weed out basic bots, but they can be easily circumvented depending on how the honeypot is implemented. Regardless, adding a honeypot still provides an additional layer of defense against bot inputs and will help deter some bot traffic on your site.

    \n

    Lockout Time:

    \n

    Often times, when bots are created to automate a task, they would be programmed to complete these tasks as quickly as possible to maximize efficiency. As a countermeasure, time lockouts can be set up to prevent bots from repeatedly spamming requests. By setting up a time lockout between requests to your domain, bots which attempt to quickly submit data on your domain will be detected. Meanwhile, human users that are registering onto the site will be working at a slower pace and will not notice the time lockout at all.

    \n

    Setting a timer on form completion does not prevent any bots from affecting your domain, but can significantly slow down their efficiency. Combined with ReCAPTCHA or other anti-bot measures, it can be very useful in reducing the impact of bots.

    \n

    Blacklisting IPs:

    \n

    If an entity accessing your website is from an unexpected location, for example, a Russian IP accessing your domain for an American service, using IP blacklists may be useful to prevent possible bot attacks. IP blacklisting can usually be set up through your hosting services and allows you to customize where users may access your domain.

    \n

    There are some issues with blacklisting, though. Choosing which targets to blacklist could be a tedious task. Even with a bot set up to detect users with suspicious activity and block them, there could be a chance of a false positive, which may result in users of your domain being blacklisted.

    \n

    Proof Of Work:

    \n

    If you want to save your site from spams and denial-of-service attacks you can incorporate a layer of Proof Of Work algorithm in your site. Whenever any client will try to connect to your server they need to commit some of their resources to the Proof Of Work algorithm first and then the server should be connected.

    \n

    With this approach, any legitimate user would experience just a negligible computational cost, but a spammer/attacker trying to establish a large number of connections would bear the computational cost and time delay, it deters the abuser to do so. There are many POW algorithms which you can use eg:- Client Puzzle Protocol, Productive Puzzle Protocol, Guided Tour Puzzle Protocol

    \n

    Other Forms of Malicious Bots:

    \n

    Other malicious bots that can impact user experience negatively include, but are not limited to, bots designed for DDOS attacks, spam bots that harvest user data, bots that create links to phishing websites which generate viruses, and malicious bot worms that infect computers.

    \n

    Countermeasures for these bots vary depending on what is being prevented. Honeypot data fields can act as a detection method against bots harvesting data, and with proper preventative training, phishing and scam bots can be handled. Third party services may also be used to protect from different forms of bots. For example, to mitigate impacts of DDOS attacks, a user may implement a solution offered by CloudFlare. Another example is the use of an ad blocker to prevent malware from being drive-by downloaded by intrusive ads.

    \n

    Conclusion:

    \n

    Unless your domain is a highly popular website, or is being targeted by technical security violation experts, there is a good chance that utilizing a simple Google ReCaptcha prompt for form completion is enough to handle any malicious bots that attempt to access your website. For domains with significantly more traffic, paid solutions like Cloudflare might also be useful in dealing with malicious bots.

    \n

    Keep in mind that although some bots are created for bad purposes, a large number of bots exist to automate beneficial tasks and make it easier for humans. Even though there are a significant amount of bots that are not helpful, it is important to embrace how useful benevolent bots actually are.

    \n","frontmatter":{"title":"A Bot Protection Overview","author":{"id":"Chris Yee","github":null,"avatar":null},"date":"May 31, 2019","updated_date":null,"tags":["Engineering","Captcha","Spam","Secure","IP"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.2578616352201257,"src":"/static/010dd09dba7a6b0a11d58d980e3f0306/14b42/Productshot.jpg","srcSet":"/static/010dd09dba7a6b0a11d58d980e3f0306/f836f/Productshot.jpg 200w,\n/static/010dd09dba7a6b0a11d58d980e3f0306/2244e/Productshot.jpg 400w,\n/static/010dd09dba7a6b0a11d58d980e3f0306/14b42/Productshot.jpg 800w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Chris Yee","slug":"/engineering/a-bot-protection-overview/"}}}]},"authorYaml":{"id":"Chris Yee","bio":"A software engineer who graduated with a Bachelor of Software Engineering from the University of British Columbia.","github":null,"stackoverflow":null,"linkedin":null,"medium":null,"twitter":null,"avatar":null}},"pageContext":{"id":"Chris Yee","__params":{"id":"chris-yee"}}},"staticQueryHashes":["1171199041","1384082988","2100481360","23180105","528864852"]}