Unlike single monolithic applications, distributed applications are dealing with multiple services. In such architectures, handling transactions could be a challenge.
Two-Phase Commit (2PC) is one of the distributed transaction strategies that we could apply.
Two-Phase Commit (2PC) Pattern
The 2PC Pattern is all about updating resources on multiple nodes in a single atomic operation.
In 2PC, it carries out an update in two phases (Figure 02).
Prepare: Each node, participating in the transaction, whether it is able to carry out an update in the second phase. Once each node is able to ensure it, the coordinator will be notified. If any of the nodes are unable to make it, the coordinator is notified to roll back releasing any locks they have with nodes.
Commit: Carrying out the update and completing the transaction.
The Saga Pattern
As explained above, your business logic can use ACID transactions within services. However, it must use Saga Pattern in order to maintain data consistency across services.
Pattern: Maintain data consistency across services using a sequence of local transactions that are coordinated using asynchronous messaging [2].
A Saga is a sequence of local transactions. Each local transaction updates the local database using the familiar ACID transaction frameworks and publishes an event to trigger the next local transaction in the Saga. If a local transaction fails, then the Saga executes a series of compensating transactions that undo the changes, which were completed by the preceding local transactions (Figure 03).
The following frameworks are available to implement the Saga orchestration pattern:
Camunda is a Java-based framework that supports the Business Process Model and Notation (BPMN) standard for workflow and process automation.
Apache Camel provides implementation for the Saga EIP (Enterprise Integration Pattern), a way to define a series of related actions in a Camel route that should be either completed successfully (all of them) or not-executed or compensated.
IBM App Connect allows you to draw out a flow using various built-in adapters and configure its properties appropriately to create a Saga flow
Some of these terms have different meanings depending on whether you’re talking about relational versus NoSQL databases.
Replication:
In always-available relational environments, you want some way to synchronize your database instances so they’re as close to up-to-date to each other as possible. This synchronization scheme is called Replication. You’ll want this for “hot failover” if your primary instance goes down, and you may have a stack of hosts that are all replicating off of a primary. You may also have “write masters” and “read slaves” that are used in very performance-sensitive environments.
Different relational DB worlds do replication differently; some directly send queries to replicas using network connections, others stream queries (or rows to be updated) as files that are “played”, etc. One concern in any replication stack is “replica lag”, which is something DBAs have to keep track of.
NoSQL clusters also have a notion of replication which is often similar in design to the idea of RAIDed disk arrays in that the same data may be “striped” to multiple nodes.
Partitioning:
Partitioning is a term that has somewhat different meanings in relational versus NoSQL worlds. In relational worlds, partitioning is a storage-level concept that applies at the table level; if one has a very large table, it can be “partitioned” into smaller storage units using various types of partitioning rules, based on a user-specified partition key. Here’s a good discussion of this type of partitioning in MySQL: What is MySQL Partitioning?
In most NoSQL worlds, partitioning describes the rules for allocating different pieces of data to different nodes, as in the vast majority of NoSQL DB’s, you don’t have all your data on every node. There is a notion of a partition key, which must be chosen with care, and is a very important part of your overall database design, and the partition key and your partitioning rules determine what nodes get a specific piece of data, and how requests for that data are routed if you’re asking for it from a node that doesn’t have that data.
The “P” in the CAP theorem is this sense of partitioning…
The analog of the NoSQL sense of partitioning in distributed relational worlds is Sharding.
Sharding:
As mentioned above, Sharding is to distributed relational database environments as Partitioning is to NoSQL environments. The main difference is that most relational databases require you to have app-visible policies for sharding, and care must be taken with the shard key so you can make sure that joins can be done within the scope of your overall shard key (without needing to attempt extremely slow cross-database joins). So, picking the right shard key is pretty fundamental to the success of a distributed relational DB world.
“Resharding” and shard-splitting are tasks that DBAs and app developers may have to deal with occasionally.
Clustering:
In relational databases, clustering unfortunately has many vendor-specific meanings. Oracle has something called a “table cluster”, which has some storage optimizations, etc. PostgreSQL has a notion of CLUSTER where individual tables can be physically rebuilt using a specific cluster key for performance reasons.
That said, the most widely-used notion of “clustering” in relational databases is probably referring to the notion of a clustered index. This is also sometimes called a primary index, in that storage engines that support clustered index use it to organize the base table data around the cluster key - which is almost always the primary key or a part of a composite PK - to maximize performance.
Replication - Copying an entire table or database onto multiple servers. Used for improving speed of access to reference records such as master data.
Partitioning - Splitting up a large monolithic database into multiple smaller databases based on data cohesion. Example - splitting a large ERP database into modular databases like accounts database, sales database, materials database etc.
Clustering - Using multiple application servers to access the same database. Used for computation intensive, parallelized, analytical applications that work on non volatile data.
Sharding - Splitting up a large table of data horizontally i.e. row-wise. A table containing 100s of millions of rows may be split into multiple tables containing 1 million rows each. Each of the tables resulting from the split will be placed into a separate database/server. Sharding is done to spread load and improve access speed. Facebook/twitter tables fit into this category.
TreeSet: The data structure for the TreeSet is TreeMap; it contains SortedSet & NavigableSet interface to keep the elements sorted in ascending order and navigated through the tree.
Why hashmap with custom key object immutable?? For sorting hashmap has customer object as key should be immutable object because of this reason string is preferred as key
Hashtable{method level locl} vs synchronisedhashMap{block level lock also supported} vs concurrenthashmap{segment lock} vs linkedhashmap vs treemap link
Heap data structure[min heap and max heap]
PriorityQueue is important data structure[default is min heap, use arraylist internally] link
Association: Simple relationship; objects can exist independently. Aggregation: A weak relationship where one object contains another but the contained object can exist independently. Composition: A strong relationship where one object contains another, and the contained object cannot exist without the container. Inheritance: A class inherits properties and behavior from another class.
JWT [Its format of token used, can be used in Authentication and authorization, auth code , accesstoken are sent as jwt token] link. Other token can be opaque token. JWT is self contained token
Claims in JWT(registered, public, private)
Client side best practice for cookie link [Since we are trying to protect our application as much as possible from hacking, we must store our refresh token exclusively in an HttpOnly Cookie.]
Sinon: Best for mocking, spying, and stubbing external dependencies in tests
Nightwatch.js: Best for simple end-to-end testing with WebDriver integration
QUnit: Best for straightforward unit testing, especially in legacy projects
Storybook: Best for visually testing and documenting UI components in isolation
Ava: Best for fast, concurrent testing with minimal configuration
Detox:Best for end-to-end testing of mobile apps, particularly React Native, with fast test execution.
Note- The algorithm React uses to diff one tree with another to determine which parts need to be changed also k/a reconciliation. .JavaScript is a dynamically typed language. In a dynamically typed language, the type of a variable is checked during run-time in contrast to a statically typed language, where the type of a variable is checked during compile-time.In JavaScript, primitive data types are passed by value and non-primitive data types are passed by reference.Functions that operate on other functions, either by taking them as arguments or by returning them, are called higher-order functions.Memoization is a form of caching where the return value of a function is cached based on its parameters. If the parameter of that function is not changed, the cached version of the function is returned.
React virtual DOM uses Fibre node as internal data structure. Its a tree of single linkedlist node
You are given 2 database table. First table named as 'goals' with fields as goal_id, player_id, team_id, goal_time.
Second table names as 'teams' with field name as team_id, team_name.
Write a sql query to fetch the name of all teams that scored atleast one goal within 30sec of time and also corresponding count count of goals with respect to teamYou are given 2 database table. First table named as 'goals' with fields as goal_id, player_id, team_id, goal_time.
Second table names as 'teams' with field name as team_id, team_name.
Write a sql query to fetch the name of all teams that scored atleast one goal within 30sec of time and also corresponding count count of goals with respect to team . Also show result in order of highest goal to lowest in descending order
Ans:
SELECT
t.team_name,
COUNT(g.goal_id) AS goal_count
FROM
goals g
JOIN
teams t ON g.team_id = t.team_id
WHERE
g.goal_time <=30
GROUPBY
t.team_name
HAVING
COUNT(g.goal_id) >=1
ORDERBY
goal_count DESC;
SELECT column1, column2, ...
INTO new_table_name
FROM existing_table
WHEREcondition;
SELECT employees.emp_name, departments.dept_name
FROM employees
JOIN departments ON employees.dept_id = departments.dept_id;
Note:
Managing large-scale databases for performance involves various strategies, including proper indexing, partitioning, query optimization, hardware optimization, and caching. Monitoring and fine-tuning the database is crucial to ensure optimal performance as data volumes grow.
Ensuring data availability and disaster recovery relies on the implementation of vital backup and recovery strategies. These strategies encompass various methods, such as full backups, differential backups, transaction log backups, and regular testing of restoration procedures.
NoSQL databases are non-relational databases designed for handling large volumes of unstructured or semi-structured data. They interact with SQL databases through various integration methods, such as data pipelines, ETL processes, and API-based data transfers
Advanced optimization techniques for SQL queries include using query hints, indexing strategies, query rewriting, and understanding the query execution plan. Profiling tools and performance monitoring are essential for identifying and resolving performance bottlenecks
The INTO clause creates a new table, not just inserts data into an existing one
Design patterns are typical solutions to commonly occurring problems in software design. They are like pre-made blueprints that you can customize to solve a recurring design problem in your code. Usage examples in java link
Creational Design pattern: are concerned with the way of creating objects. Singleton, Builder, Factory pattern, Abstract Fcatory linklink2
Structural Design pattern : These patterns explain how to assemble objects and classes into larger structures while keeping these structures flexible and efficient. Adopter, Decorator, Facade, Composite, Bridge, flyweight design pattern link
Behavioural Design pattern : are concerned with the interaction and responsibility of objects. Observer, Iterator, ChainOfResponsibility, Strategy, Command link
Microservices is an architectural style for building applications as a collection of independent services. Each service is built and deployed independently, and can be scaled independently
→ Reliable API design → Scalability and distributed systems → Database modeling and indexing → Caching strategies → Network protocols → Testing and debugging → Monitoring and observability → Version control and CI/CD → Security basics
