Blog: Posts from May, 2022

Labels
AJAX(112) Apple(1) Application Builder(242) Application Factory(207) ASP.NET(95) ASP.NET 3.5(45) ASP.NET Code Generator(72) ASP.NET Membership(28) Azure(18) Barcodes(3) BLOB(18) Business Rules(1) Business Rules/Logic(140) BYOD(13) Caching(2) Calendar(5) Charts(29) Cloud(14) Cloud On Time(2) Cloud On Time for Windows 7(2) Code Generator(54) Collaboration(11) command line(1) Conflict Detection(1) Content Management System(11) COT Tools for Excel(26) CRUD(1) Custom Actions(1) Data Aquarium Framework(122) Data Sheet(9) Data Sources(22) Database Lookups(50) Deployment(22) Designer(177) DotNetNuke(12) EASE(20) Email(6) Features(99) Firebird(1) Form Builder(14) Globalization and Localization(6) Hypermedia(2) Installation(4) JavaScript(20) Kiosk(1) Low Code(3) Mac(1) Many-To-Many(4) Maps(6) Master/Detail(36) Microservices(4) Mobile(63) Mode Builder(3) Model Builder(3) MySQL(10) Native Apps(5) News(15) OAuth(5) OAuth Scopes(1) OAuth2(6) Offline(14) Oracle(10) PKCE(1) PostgreSQL(2) QR codes(2) Rapid Application Development(5) Reading Pane(2) Release Notes(163) Reports(48) REST(26) RESTful(21) RESTful Workshop(13) RFID tags(1) SaaS(7) Security(75) SharePoint(12) SPA(5) SQL Anywhere(3) SQL Server(26) Stored Procedure(4) Teamwork(15) Tips and Tricks(81) Tools for Excel(2) Touch UI(93) Transactions(5) Tutorials(183) Universal Windows Platform(3) User Interface(331) Video Tutorial(37) Web 2.0(100) Web App Generator(101) Web Application Generator(607) Web Form Builder(39) Web.Config(9) Workflow(28)
Archive
Blog
Posts from May, 2022
Tuesday, May 17, 2022PrintSubscribe
Data Caching, RESTful API Engine, Visual Studio 2022

Code On Time release 8.9.24.0 introduces the performance enhancing Universal Data Caching and one of a kind RESTful API Engine. Visual Studio 2022 is supported in the Project Designer.

Universal Data Caching will give your applications a significant performance boost. Most databases have a few tables with the content that does not change frequently. Developers can define simple rules that will eliminate the queries to such tables. It takes only a few minutes to make your app run faster.

Specify the data controllers and and the output caching duration in the application configuration. Start reaping the benefits!

Level 3 RESTful API Engine is included now in every application built with Code On Time. The data controllers are automatically mirrored in the application API.

  • Developers can use the engine to create custom data-aware embedded pages that either blend in the Touch UI or have a completely custom presentation.
  • The engine supports OAuth 2.0 Authorization Flows suitable for the standalone web and mobile applications. Use any framework and technology to build a custom frontend with the Code On Time application in the backend.
  • External scripts and utilities can make direct calls into your application with the robust authentication based on authorization keys. Use tools like cUrl to make your application perform custom actions or change data.
  • Developers can limit the application UI to the specific user roles. For example, administrators can sign into the application and manage data while the users in other roles will only be able to sign in to confirm their identity during OAuth 2.0 Authorization.

The engine is introduced in the RESTful Workshop, the curated set of tutorials that will empower the reader to build the modern high performance software in a record time and at the fraction of the cost of other tools. Level 3 REST API is self-documented with the hypermedia embedded in the data.

RESTful API Engine makes it possible to build custom embedded pages and standalone web and mobile frontends using your application as a backend. External scripts and utilities can use your application to orchestrate the complex workflows.

Code On Time is making a big bet on the RESTful API Engine to deliver the new benefits to our customers:

  • Developers will be able to build apps serving as backends for devices that need to know the user identity and a way to read and write the user-specific data. The next release will support OAuth 2.0 Device Authorization Flow for the input-constrained devices. You may have experienced this flow when authorizing a medical device to access your health data or when allowing a smartwatch to use your music service. The same flow is authorizing a smart TV in a hotel room to access the video streaming service that you are paying for at home.
  • Cloud Identity has been on our roadmap for several years. Soon developers will be able to build a collection of apps with one of them serving as the Identity Provider while the others will become the Identity Consumers.
    • Each application will have its own security system.
    • Users will identify themselves by signing into the Identity Provider. This app may have the 2-Factor Authentication enabled or include an option to authenticate users with the third-party providers such as Google, Microsoft, or Facebook.
    • An application will become the identity consumer at runtime when the administrator creates a Cloud Identity link with another application. Device Authorization Flow will authorize the establishment of the link when approved by the administrator of the intended Identity Provider.
    • Identity Consumer application synchronizes its roles and custom OAuth 2.0 scopes with the Identity Provider database. This makes it possible to manage users of the entire application collection in one place.
    • RESTful API Engine access tokens will be issued by the Identity Provider application. The same access token will authorize requests to any application in the collection. The token will inherit the permissions configured in the Identity Provider by the collection administrator.
  • Microservices Architecture becomes a reality with Code On Time even for the smallest of developer shops. Cloud Identity and access token portability across the collection of applications make it possible to create the frontend that will exchange data with multiple backend applications. Each backend provides a custom API, has its own performance characteristic, and can be deployed independently. This improves the stability of the frontend and its overall performance. Learn more in the Introduction to Microservices Architecture.
  • The upcoming built-in GraphQL processor will parse the queries and resolve them by executing the RESTful API Engine requests. GraphQL has emerged at Facebook as the query language for the complex collections of APIs. Their numerous apps needed small subsets of data from multiple sources, which required several requests fetching too much data. Graph QL server is the “fat client” that defines the supported query types. The server accepts the queries formulated as a graph and delegates the execution to the custom resolvers. The output of multiple resolvers is merged in the graph that matches the query.
  • The App Studio application is a part of each Code On Time installation. This local web app presently starts the product activation process by redirecting developers to the account management portal. App Studio will come with the RESTful API Engine enabled in the future releases. It will serve as the local project management portal that will replace the start page of the app generator, which will be relegated to the icon tray. It will also provide the backend to the v9 Live Project Designer. The code generator will be invoked in the command line mode by the RESTful API Engine of the App Studio in response to the developer activities in the Project Designer of live apps. This is the architectural departure from our original plan outlined in the Roadmap, but we are on the finishing line now to deliver the amazing development tools in the heads of developers.
  • Content Hub, our innovative publishing platform, creates the new content in our legacy public site. We have developed this platform for internal use. The hub agent monitors the shared drives in our corporate Google Drive and transforms the changed documents into the HTML-based Display Flow content published through the RESTful API to the account management portat. The portal feeds the content to the database of the legacy site. The following capabilities of the platform will become commercially available:
    • Content Hub Agent will monitor Google Drive, Office 365, and network folders for the content in Microsoft Word Format. The agent is a multi-platform application. Changes are pushed to the hub-enabled application built with Code On Time.
    • Content Hub Add-On for applications created with Code On Time will render the content imported by the hub agent as public pages, blog posts, documentation library, newsletters, helpdesk tickets, public forum posts, etc.
    • Add-on will provide the interactive content editor based on Touch UI. It will customize the presentation style of the live “content hub” pages. It will create tickets and community posts from within the apps built with Code On Time in the same format that is used by the hub agent. It will invoke the RESTful API of the app to read and write the hub content.

The release 8.9.24.0 includes the following features and enhancements:

  • (Framework) Upgraded the framework to jQuery.3.6.0 and the latest version of the Material Icons.
  • (AppGen) Added support for VS2022.
  • (App Gen) Develop command in the project options will activate the most recent version of Visual Studio if the solution file format version is 12.00. This will ensure that VS2022 starts when both VS2022 and 2019 are installed on the same machine.
  • The enhanced Postal Address Verification silently resolves the postal addresses with Google Geocoding. The customers in the United States can achieve a perfect resolution of each address in the USPS-approved format.
  • (Touch UI) New tag form-max-(xs|sm|md) will make the contents of the form aligned in the middle of the page.
  • (Touch UI) Input field tagged as text-style-primary will display in the "primary" color.
  • (Touch UI) Modal popups will have the header when displayed in the content pages, which have their own header hidden.
  • (DAF) Client-side cookies are created with the SameSite=Strict attribute.
  • (Framework) Access control check is performed in the correct sequence for the standard membership controllers both for reading and writing of data.
  • (Touch UI) New css rules to enable hover over the fields with the Actions lookup to ensure correct display of icons in the buttons.
  • (Touch UI) Right-clicking on the logo will open the menu panel.
  • RESTful API Engine is included in the generated apps.
  • (Framework) New js/sys/restful.js provides a thin wrapper on top of Fetch API for the RESTful API Engine.
  • (Touch UI) A click on the empty part of the app toolbar or reading pane button bar will close an visible popup menu.
  • (Touch UI) Eliminated the redundant 3rd level child Data View fetching in the form views with multi-level master-detail. This significantly reduces the load time of forms with master-detail field relationships.
  • (Touch UI) Improved stability of inline editing in multi-level master detail forms. In some situations the aggressive selection of items in DataView fields would have caused the unnecessary attempts to fetch data and server-side exceptions.
  • Universal Data Caching improves the response time of the requests to read data in Touch UI.
  • (Touch UI) The focused universal inputs are redrawn whether or not they have a value. Previously only non-empty inputs were redrawn.
  • (Touch UI) Simplified parsing of the virtual page activator.
  • Integrated codemirror-5.65.2 for the upcoming Content Hub Add-On to allow display and editing of code samples.
  • (Touch UI) Refactored the code responsible for the "scrollable" state of the system tab bar. The visual presentation goes from flat to scrolled when the scrollable content is detected and the view does not have the horizontal scrolling (grid with horizontal scrolling).
  • (Touch UI) Removed redundant CSS class definition.
  • (Framework) New $app.urlArgs() method returns an object map of the page location URL parameters of the specified argument. The argument is either a string or a hypermedia object with the 'href' property. If the parameter is not specified, then the URl of the page in the browser is used for the parameter extraction.
  • (Touch UI) Account Access grant for OAuth 2.0 Authorization Flows is now supported by the framework,
  • (Offline Sync) Improved reporting of data downloading errors during sync.
  • (Framework) ASP.NET version headers and X-Powered-By headers are not returned by the apps in the HTTP response headers.
  • (Client Framework) Server-side errors raised when the data is retrieved from the server are displayed in the notifiction bar at the bottom of the screen. This helps detect data fetching errors that may remain invisible to the developers.
  • (ASPX) New method Invoke is implemented in DataControllerService to handle all incoming requests. It allows file uploading in the apps with the aspx page model.
  • (Map) Clearing of the geo map will result in the new tooltip assigned to it.
  • (Touch UI) Fixed the exception raised when the quick find filter is cleared.
  • (App Gen) Develop command in the project options will activate the most recent version of Visual Studio if the solution file format version is 12.00. This ensures that VS2022 starts when both VS2022 and VS2019 are installed on the same machine.
  • (Designer) Action property "whenKeySelected"is correctly persisted when set to "No" in the Project Designer.
  • (Site Content) CreatedDate and ModifiedDate fields are set to the DateTime.UtcNow.
  • (CMS) Renamed "Workflow Register" option to "Site Content (recommended)". This option is available in the database connection string settings.
  • (Reports) Class Library projects are provided with the correct ReportViewer references. Previously the leading and trailing spaces have caused errors when building projects.
  • (Reports) Included the reference to the Microsoft RDL Designer for VS 2022 in the custom reports. The reference is visible to the developers in VS if the designer is not installed.
  • (Code Gen) Simplified the syntax of !String.IsNullOrEmpty in the C# source code.
  • (Code Gen) Added support for IsNullOrWhiteSpace unary code generation expression.
  • (CodeGen) Value Inequality operators are produced as natural a != b and a<>b operators in C# and VB. Previously such operators were generated as !(a==b) and Not(a = b).
  • (Code Gen) C# code string constants are now presented with "string" + "string" breaks.
  • (Code Gen) C# code placed to the previous text line the symbols +|-|*|/|% that were previously wrapped to the next line.
  • (Codegen) C# code simplification of "params System.Object[]" to "params object[]".
  • (Framework) Numeric primary keys do not cause hidden exceptions when new BLOBs are uploaded with ODP.
  • (Framework) Method $app.actionInfo(path) returns the information about the action for the specified path in the format "groupId/actionId".
  • (Touch UI) Actions rendered as "Actons" field display simple confirmations and data controller confirmations.
  • (Touch UI) Remove the preloading instructions for the fonts. Added the version of the app to the font reference in the touch-theme.css.
  • (CodeGen) Class attribute definitions in C# will have "spaces" surrounding the "=" preceding the attribute parameter values.
  • (Touch UI) Right-clicking on the logo will open the menu panel.
  • (Touch UI) Click on the empty part of the app toolbar or reading pane button bar will close an visible popup menu.
  • (Touch UI) Eliminated the redundant 3rd level child Data View refresh in the form views with multi-level master-detail.
  • (Touch UI) Improved stability of inline editing in multi-level master detail forms.
Monday, May 9, 2022PrintSubscribe
Introduction to Microservices Architecture

This is an excerpt from the upcoming Microservices Workshop.

Rationale

Let's say you want to build an online marketplace. You will need the inventory management system, product catalog, shopping cart, order management system, payment, invoicing, shipping, product reviews, and many other modules. It is tempting to design a unified database that keeps track of everything and build a monolithic app on top. It will likely become a monumental effort. A better approach would be to build the individual modules in the order of their priority.

Use Code On Time to put together an inventory management system. The built-in rich user interface will likely satisfy your needs at least in the beginning. Touch-friendly UI and the ability to run the apps in the native mode (PWA) out of the box will save a tremendous amount of time and money. The built-in RESTful API Engine automatically mirrors the user interface capabilities and is ready to be enabled to serve as the backend of the custom mobile and web applications whenever the moment arrives.

As soon as the MVP (minimal viable product) of the inventory management system is ready, have it deployed.

Create a separate database to keep track of users and roles. Put together a Marketplace Identity application on top of the database with Code On Time . The built-in security system supports the 2-Factor Authentication and optional integration with the external identity providers, such as Google, Microsoft, or Facebook. Deploy the Marketplace Identity application to the server.

Sign into the inventory management system as administrator and configure the Marketplace Identity app as its Cloud Identity Provider. From now on, users signing into the inventory management system will be redirected to confirm their identity with the Marketplace Identity application.

Suppose that the next target is the order management system. Build it to make sure that you can convert the contents of the future shopping cart into a bona fide order. You can add the required tables to the same database schema. Alternatively, set up an entirely new database and make sure that its references to the product inventory are sufficient to keep track of the order-specific information such as SKU, brand, name, and price. The order history becomes isolated from the accidental changes to the product price that would affect the historical orders. Add the membership support to the dedicated database of the order management system.

Sign into the order management system as administrator and configure the Marketplace Identity app as its Cloud Identity Provider.

Continue to build the other modules of the online marketplace. Make them independent and provide each with its own database and membership. Use Code On Time to create the data management screens automatically mirrored in the custom RESTful API of the module. Configure the Marketplace Identity for user authentication.

Use the best-of-the breed frontend libraries and frameworks to put together the product catalog and the shopping cart modules. The RESTful APIs of the modules are authenticated with the access tokens issued by the Marketplace Identity application. Your custom apps will use the OAuth 2.0 protocol to authenticate the users and obtain the tokens.

You will be able to create a cosmos of powerful microservices complete with their RESTful APIs and sophisticated user interfaces for data management. Each microservice is the module that can be evolved and continuously deployed independently.

Custom mobile and web applications will rely on the microservices for their backend functionality. JavaScript-based clients can reference the restful.js script of the identity provider application. The script provides the lightweight wrapper $app.restful on top of the Fetch API. The in parameter in the method argument can specify the name of the module that will respond to the API request.

Only Code On Time makes it possible for a single-man shop or small team of developers to put together applications that rival the software produced by the industry giants! Perfect for prototyping and everyday production use.

Individual microservices can be replaced over time with the custom implementations eliminating any possibility of a technological lock-in.
Monday, May 2, 2022PrintSubscribe
ETag and HTTP Caching

RESTful API Engine fully embraced the HTTP features that were invented to make the Internet faster.

ETag

A web resource may have an optional ETag header also known as an entity tag. Its value must be a unique identifier for the resource content. The engine sets the ETag header to the hash value of the current version of the resource. Web browsers associate the entity tag with the URL of the fetched resource automatically.

If-None-Match

Browsers specify the entity tag in the If-None-Match header when a web application makes the next attempt to get the previously fetched resource. The engine produces the response data, calculates its hash value, and compares the new ETag with the If-None-Match value in the request header. If both values match, then there is no need to transfer the resource content to the client. The engine will set the response status to 304 Not Modified and keep the response body empty. Browsers understand this status code as the instruction to use the previously fetched content.

The ETag and If-None-Match headers provide a simple mechanism to reduce the volume of the client/server traffic. This is especially important when BLOB resources are fetched by the client app repeatedly. Large photos, video, and sound clips are downloaded to the client only once if there are no changes. The subsequent requests to the same resource will result in “304” responses from the server as long as its content has not changed.

The network trace shows two GET requests fetching the list of products. The first request with the "200" status has fetched the resource data. The second request with the "304" status has fetched an empty body and had to re-use the cached data of the previous response instead.
The network trace shows two GET requests fetching the list of products. The first request with the "200" status has fetched the resource data. The second request with the "304" status has fetched an empty body and had to re-use the cached data of the previous response instead.

The performance gains are achieved through the reduced response size. The engine will still produce the resource content on the server and calculate its hash values in response to each network request from the client.

If-Match

CRUD operations may find another use for the entity tags. The entity tag specified in the If-Match header of the request will let the RESTful API Engine to detect the midair collisions. The engine will figure the current ETag of the resource when PUT, POST, PATCH, or DELETE are specified as the HTTP method of the request. If the values of ETag and If-Match are not equal, then the error with the 412 Precondition Failed status is returned. Simply put, the error means that another user has changed the resource after it was fetched by the client app.

The value of the ETag header is automatically copied to the etag property of the _links.self hypermedia control in the resource fetched with the $app.restful method.

Client apps can include the etag property in the method argument when changing data or invoking custom actions. If the original resource content with the modified fields is specified in the body property of the $app.restful method argument, then set the etag to true for conflict detection. Otherwise set the etag value to the _links.self.etag property of the previously fetched resource.

Here is an example of the code that will demonstrate the midair collision.

JavaScript
1234567891011121314151617181920212223242526272829303132333435window.addEventListener('load', e => {
    document.querySelector("h1").addEventListener("click", e => {
        // fetch the arbitrary product resource
        $app.restful({
            url: '/v2/products/17'
        }).then(product => {
            // change the unitPrice
            product.unitPrice = product.unitPrice + 0.1;
            // **** "Edit" Attempt #1 *****
            // Send the modified body to the URL specified in
            // the "edit" hypermedia control to PATCH the resource.
            $app.restful({
                url: product._links.edit,
                body: product,
                etag: true
            }).then(changedProduct => {
                // alert the user to the new UnitPrice
                alert(
                    'The new price of "' + changedProduct.productName +
                    '" is ' + changedProduct.unitPrice);
                // The second attempt to patch the product resource will fail 
                // since the etag in "product._links.self.etag" does not match
                // the ETag of the changed product resource on the server.
                product.unitPrice = product.unitPrice + 0.1;
                // **** "Edit" Attempt #2 *****
                $app.restful({
                    url: product._links.edit,
                    body: product,
                    etag: true
                })
                    .catch(restfulException);
            });
        });
    });
});

Add the code to the bottom of the closure in the spa4.js file of the SPA4 client app. The code will run when users click the h1 element showing the name of the app at the top of the page.

The following sequence of events will take place:

  1. User clicks the header of the app.
  2. The product with the primary key specified in the url parameter of the $app.restful method argument is fetched from the /v2/products/17 resource (see the sample response data).
  3. The unitPrice property of the product variable is increased by $0.01. The variable is pointing to the object fetched in the previous step.
  4. The first attempt to edit the resource is performed. The result of the edit is the new state of the resource in the changedProduct argument. The alert will show the name of the changed product and its unitPrice.
  5. The unitPrice property of the same product variable is increased again by the same amount.
  6. The second attempt to edit the resource is performed with the body property in argument still pointing to the product variable. The $app.restful method will use the “old” entity tag extracted from the variable as the value of the If-Match header. The request fails since the ETag of the resource on the server does not match.

The "GET" request fetches the resource from the server. The first "PATCH" request changes the resource field successfully. The second "PATCH" request fails with the "412 Precondition Failed" status due to the midair conflict.
The "GET" request fetches the resource from the server. The first "PATCH" request changes the resource field successfully. The second "PATCH" request fails with the "412 Precondition Failed" status due to the midair conflict.

The screenshot shows the alert displayed after the first attempt to edit the product.

image9.png

The error message will be reported when the alert is closed since the second attempt to edit the product has failed. The engine will recommend specifying the different entity tag in the If-Match header.

image6.png

Both attempts to edit the product will succeed if the etag property is removed from the $app.restful argument of the second attempt.

ETag is the version number of the resource. Entity tag makes it possible to detect conflicts when specified in the If-Match header.

A better approach is to use the resource that was fetched during the first attempt to edit the product. Replace references to the product variable with the changedProduct and keep etag property for robust conflict detection.

JavaScript
1234567891011121314151617181920212223242526// fetch the arbitrary product resource
$app.restful({
    url: '/v2/products/17'
}).then(product => {
    // change the unitPrice
    product.unitPrice = product.unitPrice + 0.1;
    // **** "Edit" Attempt #1 *****
    $app.restful({
        url: product._links.edit,
        body: product,
        etag: true
    }).then(changedProduct => {
        // alert the user to the new UnitPrice
        alert(
            'The new price of "' + changedProduct.productName +
            '" is ' + changedProduct.unitPrice);
        changedProduct.unitPrice = changedProduct.unitPrice + 0.1;
        // **** "Edit" Attempt #2 *****
        $app.restful({
            "url": changedProduct._links.edit,
            "body": changedProduct,
            "etag": true
        })
            .catch(restfulException);
    });
});

If the body property is not referencing the original resource fetched with the $app.restful method, then set the etag property explicitly. This is the example of the ad-hoc body parameter with the reference to the etag value.

JavaScript
123456789101112131415161718192021222324252627// fetch the arbitrary product resource
$app.restful({
    url: '/v2/products/17'
}).then(product => {
    // change the unitPrice
    product.unitPrice = product.unitPrice + 0.1;
    // **** "Edit" Attempt #1 *****
    $app.restful({
        url: product._links.edit,
        body: product,
        etag: true
    }).then(changedProduct => {
        // alert the user to the new UnitPrice
        alert(
            'The new price of "' + changedProduct.productName +
            '" is ' + changedProduct.unitPrice);
        // **** "Edit" Attempt #2 *****
        $app.restful({
            "url": changedProduct._links.edit,
            "body": {
                unitPrice: changedProduct.unitPrice + 0.1
            },
            "etag": changedProduct._links.self.etag
        })
            .catch(restfulException);
    });
});

Conflict detection requires a certain amount of work on the server. Do not specify the etag parameter when invoking the $app.restful method if the simultaneous editing of the resources in the client apps is not expected.

Cache-Control

The combination of the ETag and If-None-Match headers allows avoiding the redundant transfer of the resource data to the client. The elimination of the GET requests to the previously fetched resources will provide an enormous performance boost both to the client app and the backend application.

Universal Data Caching

The framework in the foundation of the applications created with Code On Time implements the universal data caching. The simple rules provided by developers can instruct the app to keep some of the fetched data in the application cache for the specified duration. The cached data is shared by the entire user base. This eliminates the database interactions and significantly reduces the response time of the backend application.

This is the example of the universal caching rules directing the Data Aquarium framework to keep the responses from the Suppliers and Categories controllers in the cache for 1 minute. The rules are defined in ~/app/touch-settings.json configuration file.

JSON
12345678910{
  "server": {
    "cache": {
      "rules": {
        "suppliers|categories": {
          "duration": 1
        }
      }
    }
}

By default, only the private caching of the RESTful resources is allowed on the client. Entity tags coupled with the 304 Not Modified status code instruct the clients to reuse the previously fetched data.

The built-in RESTful API Engine piggybacks on the same rules to cache the responses associated with the specific URLs in the application cache on the server. Furthermore it instructs the clients to keep the responses in their own cache for the same duration by including the max-age parameter in the Cache-Control header.

The engine navigates the segments of the requested URL and figures the data controller that will be producing the response. If there are caching rules, then the engine will try to locate the response in the cache. If the previous response is not available, then the physical execution of the request will take place with the subsequent placement to the server cache. In either case the max-age parameter is specified in the Cache-Control header.

The caching of the RESTful resource greatly reduces the response time of the fist fetch by the client. The max-age parameter eliminates the further requests completely for the specified duration.

The single page application SPA4 with CRUD fetches the lookup resources of categories and suppliers when the Edit Product form is displayed.

image7.png

The network log shows the requests executed by the client after the following sequence of user actions:

Firefox Developer Tools provide a complete list of HTTP requests executed by the web application. Other browsers may not provide the full picture of the network traffic between the client and the server.
Firefox Developer Tools provide a complete list of HTTP requests executed by the web application. Other browsers may not provide the full picture of the network traffic between the client and the server.

  • STEP 1: Click the Aniseed Syrup product in the product list.
    • The OPTION request is performed by the browser to validate the CORS policy of the backend application.
    • The /v2/products/3 resource is fetched with the response code 200.
    • The Edit Product form is constructed, which causes one PATCH request preceded by the OPTIONS check. The application generates the form using the metadata reported by the RESTful API Engine. The form layout is reused in the subsequent request.
    • A request to GET the suppliers fetches the SupplierID lookup data. It is preceded by the OPTIONS validation request.
    • A request to GET the categories fetches the CategoryID lookup data. It is also preceded by the OPTIONS validation request.
  • Click the Cancel button to close the form.
  • STEP 2: Click the same product row the second time.
    • The /v2/products/3 resource is fetched with the response code 200. The previous PATCH request to this resource has invalidated the entity tag in the view of the browser. The resource data is fetched again even though the data has not changed on the server.
    • Suppliers are fetched with the 200 status code directly from the client cache.
      max-age | local cache
    • Categories are fetched with the 200 status code directly from the client cache.
      max-age | local cache
  • Click the Cancel button to close the form.
  • STEP 3: Click the Aniseed Syrup product row the third time.
    • The /v2/products/3 resource is fetched from cache after the 304 status code is reported by the server indicating that the content has not changed. The request was processed on the server but no data was transmitted to the client.
      GET | 304 | local cache
    • Suppliers are fetched with the 200 status code directly from the client cache.
      max-age | local cache
    • Categories are fetched with the 200 status code directly from the client cache.
      max-age | local cache
  • Click the Cancel button to close the form.
  • STEP 4: Click the Alice Mutton product in the product list.
    • The /v2/products/17 resource is fetched with the response code 200. It is preceded by the OPTIONS validation request.
    • Suppliers are fetched with the 200 status code directly from the client cache.
      max-age | local cache
    • Categories are fetched with the 200 status code directly from the client cache.
      max-age | local cache
  • Click the Cancel button to close the form.
  • STEP 5: Wait for 30 minutes and click the Alice Mutton product the second time.
    • The /v2/products/17 resource is requested but not fetched. The status code of the response is 401 Unauthorized. The access token has expired.
    • Method $app.restful performs automatic refresh of the access token by executing the POST request to the /oauth2/v2/token resource.
    • Method $app.restful tries again to request the /v2/products/17 resource with a new access token. The resource is fetched with the response code 200.
    • Suppliers are fetched from cache after the 304 status code is reported by the server indicating that the resource content has not changed.
      GET | 304 | local cache
    • Categories are fetched from cache after the 304 status code is reported by the server indicating that the resource content has not changed.
      GET | 304 | local cache
  • Click the Cancel button to close the form.
  • STEP 6: Click the Alice Mutton product the third time.
    • The /v2/products/17 resource is fetched from cache after the 304 status code is reported by the server indicating that the resource content has not changed.
      GET | 304 | local cache
    • Suppliers are fetched with the 200 status code directly from the client cache.
      max-age | local cache
    • Categories are fetched with the 200 status code directly from the client cache.
      max-age | local cache

Web applications are taking full advantage of the HTTP Caching when working with the RESTful API Engine of applications created with Code On Time. Native client apps may also mirror the behavior of the browsers by inspecting the ETag and Cache-Control headers of responses and specifying the previously fetched entity tags in the If-None-Match header of requests.

Public API Key

If an API Key is specified in the URL and caching is enabled, then the Cache-Control header is set to the public value. It allows responses to be stored in the shared caches on the way to the client.

Here is the sample item Baked Goods from the categories collection resource. It includes the API Key aptly named “public” in the self hypermedia control and in the value of the picture field.

JSON
1234567891011121314{
    "_links": {
        "self": {
            "href": "/v2/public/categories/217"
        }
    },
    "categoryId": 217,
    "categoryName": "Baked Goods",
    "description": "Breads, cookies, pastries",
    "picture": "/v2/public/categories/MjE3L3BpY3R1cmUvag.jpeg",
    "fileName": "bread.jpeg",
    "contentType": "image/jpeg",
    "length": 13155
}

The API Key allows loading the resource directly in the browser window.

image5.png

Perform the following sequence of actions while monitoring the requests:

  • STEP 1: Open a new browser window, open Developer Tools, and enter the full URL of the picture directly in the address bar to see the /v2/public/categories/MjE3L3BpY3R1cmUvag.jpeg resource. Note that you file name may be different than the one in this example.
    • The image is fetched with the GET request.
      GET | 200
  • STEP 2: Place cursor in the browser address bar and press Enter key
    • The image is fetched from the local private cache. The network request is not performed.
      max-age | local cache
  • STEP 3: Place cursor in the browser address bar and press Enter key one more time.
    • Once more the image is fetched from the private cache without any help from the server.
      max-age | local cache
  • STEP 4: Click the Refresh button on the left-hand side of the address bar in the browser.
    • The browser makes a GET request to the server with the If-None-Match header set to the ETag of the image in the local cache.
    • The server responds with the 304 Not Modified status and nothing in the body.
    • The image is fetched from the local cache again.
      GET | 304 | local cache

Each successful request to GET the image is followed by a 404 error. Most browsers will attempt to load the "favico.ico" when an URL is entered in the address bar. The 404 response simply indicate that there is no icon in the application root on the server.
Each successful request to GET the image is followed by a 404 error. Most browsers will attempt to load the "favico.ico" when an URL is entered in the address bar. The 404 response simply indicate that there is no icon in the application root on the server.

RESTful API Engine will include the max-age and s-maxage parameters in the Cache-Control header if the path or query parameter specifies an API Key. The first parameter will tell the browsers to keep the response in the private cache. The second parameter will tell the shared caches and proxies to keep the public response for the specified duration. This will completely eliminate the handling of the incoming requests by the application server, since the shared cache server will have the response ready for the clients.

JSON and BLOB responses to the requests with the API keys  specified in the URLs are configured for public caching by the browsers, shared caches, and proxies.
JSON and BLOB responses to the requests with the API keys specified in the URLs are configured for public caching by the browsers, shared caches, and proxies.

Embedding With Caching

RESTful API Engine does not suffer from the under-fetching problem typical to the Level 2 REST APIs and below. Developers can fetch multiple resources in a single request thanks to the hypermedia.

Consider the products collection with the limit of 1 item presented below. The links supplierId and categoryId in the one and only item in the collection are marked with the embeddable property.

JSON
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061{
    "_links": {
        "self": {
            "href": "/v2/products?limit=1"
        },
        "first": {
            "href": "/v2/products?page=0&limit=5"
        },
        "lookup-supplierId": {
            "href": "/v2/suppliers?count=true&fields=supplierId,companyName",
            "embeddable": true
        },
        "lookup-categoryId": {
            "href": "/v2/categories?count=true&fields=categoryId,categoryName",
            "embeddable": true
        },
        "create": {
            "href": "/v2/products",
            "method": "POST"
        },
        "do-something": {
            "href": "/v2/products/do-something",
            "method": "POST"
        },
        "report1": {
            "href": "/v2/products/report1"
        },
        "schema": {
            "href": "/v2/products?_schema=true&limit=1"
        }
    },
    "collection": [
        {
            "_links": {
                "self": {
                    "href": "/v2/products/17"
                },
                "supplierId": {
                    "href": "/v2/products/17/supplier-id",
                    "embeddable": true
                },
                "categoryId": {
                    "href": "/v2/products/17/category-id",
                    "embeddable": true
                }
            },
            "productId": 17,
            "productName": "Alice Mutton",
            "supplierId": 7,
            "supplierCompanyName": "Pavlova, Ltd.",
            "categoryId": 6,
            "categoryName": "Meat/Poultry",
            "quantityPerUnit": "20 - 1 kg tins",
            "unitPrice": 16.9000,
            "unitsInStock": 97,
            "unitsOnOrder": 23,
            "reorderLevel": 5,
            "discontinued": false
        }
    ]
}

Modify the URL of the request to have the _embed parameter with the value that enumerates the embeddable resources. Also add the _links=false parameter to remove the hypermedia controls from the output

/v2/products?limit=1&_embed=supplierId,categoryId&_links=false

The supplierId and categoryId field values in the collection item of the response are objects with the properties of the embedded resources. The engine automatically replaces the original fields with the corresponding embeddable resources.

JSON
1234567891011121314151617181920212223242526272829303132333435363738394041{
    "collection": [
        {
            "productId": 17,
            "productName": "Alice Mutton",
            "supplierId": {
                "supplierId": 7,
                "companyName": "Pavlova, Ltd.",
                "contactName": "Ian Devling",
                "contactTitle": "Marketing Manager",
                "Products2": "/v2/products/17/supplier-id/products2",
                "address": "74 Rose St. Moonie Ponds",
                "city": "Melbourne",
                "region": "Victoria",
                "postalCode": "3058",
                "country": "Australia",
                "phone": "(03) 444-2343",
                "fax": "(03) 444-6588",
                "products": "/v2/products/17/supplier-id/products"
            },
            "supplierCompanyName": "Pavlova, Ltd.",
            "categoryId": {
                "categoryId": 6,
                "categoryName": "Meat/Poultry",
                "description": "Prepared meats",
                "picture": "/v2/products/17/category-id/picture",
                "fileName": null,
                "contentType": null,
                "length": null,
                "products": "/v2/products/17/category-id/products"
            },
            "categoryName": "Meat/Poultry",
            "quantityPerUnit": "20 - 1 kg tins",
            "unitPrice": 16.9000,
            "unitsInStock": 97,
            "unitsOnOrder": 23,
            "reorderLevel": 5,
            "discontinued": false
        }
    ]
}

Take a note of the time it took to produce the response in a tool like Postman. Try execution of the same request a few times in the row. The response time will decrease thanks to the caching of Suppliers and Categories.

Remove the limit=1 parameter from the URL of the request and take a note of the time it takes to complete. About 80 product items will be included in the collection with the supplierId and categoryId fields resolved. Subsequent requests to the same URL are executing up to 40 times faster.

Add the Products data controller to the caching rules and you will discover further improvement in the response time, which was up to 200 times faster compared to the app configuration without caching in our own test.

Learn about fetching the data in the exact shape that is needed for the client app in the No More Under-Fetching Or Over-Fetching segment of the RESTful Workshop.

Latest Version

HTTP Caching does improve performance but makes it impossible to manage the data in the client apps. Universal Data Caching supports the exemptions. Developers can specify the roles that will not be affected by the caching rules. If the identity of the user associated with the access token or API Key has the role or scope of an exemption, then the engine will return the current version of the resource.

RESTful API Engine provides the means of reading the latest version of resources by users that are not exempt from caching.

Consider the categories collection resource in the next snippet.

The hypermedia control self has the max-age property letting the developers know that there is the caching rule set for this resource. The caching duration is 60 seconds. There is also the latest-version hypermedia control that provides access to the latest version of the resource content.

JSON
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465{
    "_links": {
        "self": {
            "href": "/v2/categories",
            "max-age": 60
        },
        "latest-version": {
            "href": "/v2/categories/latest-version"
        },
        "first": {
            "href": "/v2/categories?page=0&limit=10"
        },
        "create": {
            "href": "/v2/categories",
            "method": "POST"
        },
        "schema": {
            "href": "/v2/categories?_schema=true"
        }
    },
    "collection": [
        {
            "_links": {
                "self": {
                    "href": "/v2/categories/217"
                }
            },
            "categoryId": 217,
            "categoryName": "Baked Goods",
            "description": "Breads, cookies, pastries",
            "picture": "/v2/categories/MjE3L3BpY3R1cmUvag.jpeg",
            "fileName": "bread.jpeg",
            "contentType": "image/jpeg",
            "length": 13155
        },
        {
            "_links": {
                "self": {
                    "href": "/v2/categories/1"
                }
            },
            "categoryId": 1,
            "categoryName": "Beverages",
            "description": "Soft drinks, coffees, teas, beers, and ales",
            "picture": "/v2/categories/1/picture",
            "fileName": null,
            "contentType": null,
            "length": null
        },
        {
            "_links": {
                "self": {
                    "href": "/v2/categories/2"
                }
            },
            "categoryId": 2,
            "categoryName": "Condiments",
            "description": "Sweet and savory sauces, relishes, spreads, and seasonings",
            "picture": "/v2/categories/2/picture",
            "fileName": null,
            "contentType": null,
            "length": null
        }
    ]
}

Here is the latest version of the categories collection resource. Note the latest-version segment is in the path of the hypermedia links. This magic suffix in the URL path will create a caching exemption that overrides the server caching rules.

JSON
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061{
    "_links": {
        "self": {
            "href": "/v2/categories/latest-version"
        },
        "first": {
            "href": "/v2/categories/latest-version?page=0&limit=10"
        },
        "create": {
            "href": "/v2/categories/latest-version",
            "method": "POST"
        },
        "schema": {
            "href": "/v2/categories/latest-version?_schema=true"
        }
    },
    "collection": [
        {
            "_links": {
                "self": {
                    "href": "/v2/categories/217/latest-version"
                }
            },
            "categoryId": 217,
            "categoryName": "Baked Goods",
            "description": "Breads, cookies, pastries",
            "picture": "/v2/categories/latest-version/MjE3L3BpY3R1cmUvag.jpeg",
            "fileName": "bread.jpeg",
            "contentType": "image/jpeg",
            "length": 13155
        },
        {
            "_links": {
                "self": {
                    "href": "/v2/categories/1/latest-version"
                }
            },
            "categoryId": 1,
            "categoryName": "Beverages",
            "description": "Soft drinks, coffees, teas, beers, and ales",
            "picture": "/v2/categories/1/picture/latest-version",
            "fileName": null,
            "contentType": null,
            "length": null
        },
        {
            "_links": {
                "self": {
                    "href": "/v2/categories/2/latest-version"
                }
            },
            "categoryId": 2,
            "categoryName": "Condiments",
            "description": "Sweet and savory sauces, relishes, spreads, and seasonings",
            "picture": "/v2/categories/2/picture/latest-version",
            "fileName": null,
            "contentType": null,
            "length": null
        }
    ]
}

If the hypermedia controls are used in the client application, then it is trivial to switch between the cached and the latest version of a resource. It is easy to know when the cached data is being served by the backend application.

If the max-age parameter is specified in the self hypermedia control, then the data is cached. Use the latest-version control to access the latest resource content.