You are reading the article Learn The Types And Methods Of Mapreduce Api updated in December 2023 on the website Achiashop.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested January 2024 Learn The Types And Methods Of Mapreduce Api
Introduction to MapReduce APIHadoop can be developed in programming languages like Python and C++. MapReduce Hadoop is a software framework for ease in writing applications of software processing huge amounts of data. MapReduce is a framework which splits the chunk of data, sorts the map outputs and input to reduce tasks. A File-system stores the output and input of jobs. Re-execution of failed tasks, scheduling them, and monitoring them is the task of the framework. We will learn how to write a code in Hadoop in MapReduce and not involve Jython to translate code into Java. In this topic, we are going to learn about the MapReduce API.
Start Your Free Data Science Course
Hadoop, Data Science, Statistics & others
MapReduce ClassHere are the following MapReduce Class
1. Map PhaseThe map phase splits the input data into two parts. They are Keys and Values. Writable and comparable is the key in the processing stage where only in the processing stage, Value is writable. For example, let’s say a client gives input data to a Hadoop system; the task tracker is assigned tasks by the job tracker.
Mini reducer, which is commonly called a combiner, the reducer code places input as the combiner. Network bandwidth is high when a huge amount of data is required. Hash is the default partition used. The partition module plays a key role in Hadoop. More performance is given by reducing the pressure of the petitioner on the reducer.
2. Processing in Intermediate 3. Reducer PhaseThe reducer takes in the data input that is sorted and shuffled. All the input data is going to be combined, and similar key-value pairs are to be written to the hdfs system. For searching and mapping purposes, a reducer is not always necessary. Setting some properties for enabling to set the number of reducers for each task. During job processing, speculative execution plays a prominent role.
Types of methods in MapReduce APILet us see all the classes and methods are provided for MapReduce programming.
1. Mapper Class of MapReduceMethods:
void map
(VALUEIN value, KEYIN key, Context context)
(Context context)
(Context context)
(Context context)
This method is called for every key-value pair in the input split and only at once.
This method is called at the end of the task and only at once.
This method is called at the beginning of the task and only at once.
This method overrides to control the Mapper execution.
2. Reducer Class of MapReduceMethods:
void map (Context context)
(Context context)
(Context context)
This method is called for each key and only once.
This method is called only at the beginning of the task and only at once.
This method is used to control all the Reducer tasks.
3. Job Class of MpReduceMethods:
String getJobFile()
(Configuration conf, String JobName)
(Configuration conf)
Job getInstance()
long getFinishTime()
Counters getCounters()
This method is used for setting the Reducer for jobs.
This method is used for setting the number of Reducers for jobs.
This method is used to set the job of the Mapper.
This method is used for mapper output data to set the value class.
The user-specified job name is to be set by using this method.
Providing the class name with the extension .class, this method is used to set the jar.
This method is used to get the job function with a schedule.
The user-specified job name is obtained by using this method.
The path of the submitted configuration of the job is obtained by using this method.
Generates a new Job without a cluster and with provided job name and configuration.
Generates a new Job without a cluster and with provided job name and configuration.
New Job with no cluster.
Finish time of Job.
Counters of the job.
Understanding how to submit a job with an example:
ConclusionImagine you have lots of documents, which is huge data. And you need to count the number of occurrences of each word throughout the documents. It might seem like an arbitrary task, but the basic idea is that let’s say you have a lot of web pages, and you want to make them available for search queries. Aggregation of data is done by the reducer, and it consists of all the keys and combines them all for similar key-value pairs, which is basically the Hadoop shuffling process.
Recommended ArticlesThis is a guide to MapReduce API. Here we discuss the MapReduce Class along with the Types of methods in MapReduce API. You may also have a look at the following articles to learn more –
You're reading Learn The Types And Methods Of Mapreduce Api
Learn The Various Types Of Microcomputers
Introduction to Microcomputer Types
Web development, programming languages, Software testing & others
List of Microcomputer typesThere are different Microcomputer types given below.
Desktop Computer
Laptop
Smartphone
Notebook
Tablet
1. Desktop Microcomputer 2. Microcomputer LaptopA laptop, also known as a “Notebook,” is a computer that looks like a sleek briefcase. They can also handle more sophisticated tasks, such as a desktop computer, and they can run on their own in-built battery and a wall socket. Since different small-size components, including a keyboard, LCD monitor, touchpad, and other internal parts, are incorporated into these microcomputers, they are more costly than desktop computers (CPU, Motherboard, Hard disk, etc.). However, its greatest benefit is its portability.
3. Tablets and SmartphonesTablets
Tablet microcomputers are compact mobile computers with touch screen displays that are smaller than laptop computers but larger than smartphones. On the LCD panel, users can perform both input and output operations. In addition, they can access the internet through Wi-Fi and other cellular networks. Users can experience issues due to a lack of a keyboard, but they may use an external keyboard if more is needed. Tablets such as Apple’s iPad, Microsoft’s Surface, and Amazon’s Kindle Fire are commercially available.
Smartphones
PDA is short for “Personal Digital Assistant,” and it is a portable device that is equivalent to a tablet, palmtop, or smartphone in terms of portability and scale. It has a small LCD screen for both input and output functions. a PDA can communicate with other computers such as desktops, laptops, and other devices Via infrared (IR), LAN cable, Bluetooth, Wi-Fi, radio waves, and wireless beams. PDAs are mostly used to keep track of to-do lists, appointment schedules, address books, and to take notes.
4. Palmtop microcomputerThe term “pocket computer” refers to a palmtop microcomputer. It is a smaller computer than a PDA, and it uses less power as a result of its compact scale. It is also a very lightweight laptop. Instead of using a keyboard, palmtop computers use a stylus pen stick to enter data. Palmtops don’t usually have disk drives because of their limited size, so they rely on PCMCIA slots to allow disk drives, multiple modems, memory management, and another terminal. Palmtop computers and other portable computers usually run Windows CE operating systems.
Workstation Microcomputers
Workstation microcomputers, also known as efficient one-side computers, are multiprocessor computers with several CPUs. Typically, this microcomputer is programmed by a single user to run specific programs that need more power than a standard PC.
The server microcomputer looks like a computer, but it has a number of powerful features that enable it to render data to another terminal over a variety of networks, including local area networks (LANs) and wide-area networks (WANs) (WAN). Any type of server is programmed to accomplish a particular task, such as running the Apache HTTP server to manage all websites on the internet and the Mail server to send and receive emails.
Mini Tower Microcomputer
The mini-tower microcomputer was created in the shape of a small tower. Due to its small scale, it can be mounted on the table with less space than other desktop microcomputers. In addition, different buses may be used to link input/output devices to it.
Full Tower Microcomputer
Recommended ArticlesThis is a guide to Microcomputer types. Here we discuss the various types of microcomputers in detail with their specifications and features. You may also look at the following articles to learn more –
Techniques And Types Of Storage Virtualization
Introduction to Storage Virtualization
Hadoop, Data Science, Statistics & others
Types of Storage VirtualizationWidely these storage systems provide two kinds of access to our system either block or file-based access.
So we can broadly classify our storage virtualization as:
1. Block VirtualizationIn block virtualization, we basically separate our logical storage from that of the physical so that the user/administrator can access without having to access the physical storage, basically doing this way helps the administrator in giving a lot of flexibility in managing different storage.
2. File VirtualizationIn File virtualization, it basically removes the dependencies caused in accessing the data at file level to that of the location where they are actually present. This basically helps in overcoming the challenges faced with network-attached storage and they also help in optimizing the storage usage and also help us to do some file migrations in a non-disruptive way.
Methods of VirtualizationVirtualization typically refers to the pooling of different available storage and maintaining them in single storage in virtual environment, recent technologies such as hyper-converged infrastructure makes use of not only virtual storage but also power and network as well.
1. Host-Based Virtualization Approach 2. Array-Based Virtualization Approach 3. Network-Based Virtualization Approach Configurations of Storage VirtualizationIdeally, we have two different ways in configuring our storage virtualization they are:
1. In-Band Approach (Symmetric)In this method, we store the virtual environment configuration in the data path itself as in the data as well as the control flow. This kind of solution is considered easy/simple to implement as we do not use any kind of software. We do different levels of abstraction inside the data path. These kinds of solutions help us to improve our device’s performance majorly and also prolong the useful life of the devices. One of the examples of an in-band based solution is that IBM’s total storage area network volume controller.
2. Out-Band Approach (Asymmetric)In this approach, the implementation of the virtual environment is done outside of the data path as in the data flow and the control flow are separated which can be achieved by separating our Metadata from data and putting them in different places. This kind of virtualization involves in transferring all the tables to a Metadata controller which has all the Metadata files. By separating both the flows we achieve the usage of complete bandwidth that is offered by the storage area network.
Benefits of Storage VirtualizationNow that we have seen what is storage virtualization and its types and also how do we implement them, now let us see some of the benefits of going to storage virtualization:
Our data does not get compromised easily even if the host fails as we store our data in a different and convenient place.
It is easy for us to protect, provide and use our data as we implement some level of abstraction in our storage.
Additional functions such as recovery, duplication, replication, etc. can be done with ease.
ConclusionBy now from the article you would have understood quite a lot about storage virtualization and its techniques, pros, and cons. It is necessary for us to go towards these kinds of virtualization approaches as it helps in reducing the complexity of the way data is stored and also helps the storage administrator in performing tasks such as disaster recovery, backup or archival of data easily in less amount of time.
Recommended ArticlesThis is a guide to Storage Virtualization. Here we discuss a brief overview of storage virtualization and the different types, methods, benefits, etc. You can also go through our other suggested articles to learn more –
Learn The Working Of The Jmeter Tool
Introduction to JMeter Tool
Performance testing is important in the application and software development life cycle. For example, if many users use websites, APIs, or applications, they should plan how to manage peak traffic over extended periods. There are many tools available in the market to perform load testing. The popular tool is Apache JMeter, which performs a load test on software and web services like REST and SOAP. This article can briefly examine JMeter, its working, configuration, and other features.
Start Your Free Software Development Course
JMeter Tool
JMeter is a popular tool available as open-source and based on Java. It is a performance testing tool that executes protocol-oriented load tests again different software and web services, API, web applications, SOAP, REST, and other web services. Despite being introduced earlier, JMeter remains a powerful tool among other load testing and performance solution tools due to its reliable features.
In simple, JMeter operates by aping visitors to the service or application, enabling the user to create and transfer HTTP requests to the server. Then the statistical data is displayed by collecting the server response data in tables, charts, and reports. From the generated reports, the user can analyze the application’s behavior, perform bottlenecks, and tweak the website to enhance its efficacy or performance; sometimes, the application improves by offering a few productive insights.
It is mandatory to see that JMeter is not a web browser; it functions at a protocol level and can’t support all the changes performed by the user, like rendering JavaScript. Apart from this, JMeter offers temporary solutions for these issues by providing configuration elements like cookie manager, header manager, and cache manager, which support JMeter’s behavior and function as an actual browser.
JMeter is a desktop application based on Java, so to begin this, there are a few perquisites where it should be configured to initiate the performance test. Consider if the user has the latest Java version installed, which should support the minimum system requirements to use JMeter. Compared to other software as a service load testing tool, it consumes additional resources and time.
JMeter is the most common tool by developers in software development, where their team is comfortable executing the load performance tests. JMeter has gained endurance in the market due to its strong community support, extensive documentation, and wide range of best practices available. In addition, it is cost-effective and can be implemented in an enterprise requiring no or minimal resources to get started.
JMeter is free to use and enables the developers to use the source code for development, testing, and training purposes.
JMeter has a friendly graphical user interface that beginners find easy to use and doesn’t require extra time to understand the tool. It has a simple and direct interface.
JMeter is platform-dependent, based on Java, and can be executed on multiple platforms.
The complete multithreading framework allows the JMeter to simultaneously sample varied functions by a distinct thread group. As a result, it increases the concurrency even in multiple thread groups.
It helps to visualize the output result. The test result can be viewed in tables, trees, charts, and log files.
JMeter installation is easy; the user can copy and execute the *bat file into the host machine.
JMeter is highly extensible as it enables the user to compose his test. In addition, it supports plugins to make effective visualization to extend the testing.
JMeter supports multiple protocols, which support testing on web applications and compute performance tests on database servers. JMeter supports all the basic protocols like JDBC, HTTP, JMS, SOAP, LDAP, FTP, and REST.
The record and playback option is one of the remarkable features of JMeter, where the user can record the activities on the browser and produce them on the website using JMeter. Additionally, JMeter allows for seamless integration of testing and scripting capabilities with Selenium and Beanshell, enabling automated testing functionalities.
Working on JMeter Tool
The working of JMeter is simple, and it follows only a few steps to execute the load testing on applications. First, the JMeter makes multiple users send requests to the target server, extracting the performance result from the target server.
JMeter creates multiple requests to the target server and waits for feedback.
Once the server responds to JMeter, it stores and processes all the responses.
The system computes and collects the responses and generates statistical information or a report.
Then the generated report is fed again as a request to the target server.
You can obtain the analytical report in charts, graphs, tables, or trees.
ConclusionAs the rise of software as a service-based tool becomes predominant and rapid technological changes are consistent, the use of JMeter has never faded. JMeter provides the best tool for load testing solutions, and it comes as a package with all the cloud-based solutions with all the support, benefits, and features and is used as a monthly scheme.
Recommended ArticlesWe hope that this EDUCBA information on “JMeter Tool” was beneficial to you. You can view EDUCBA’s recommended articles for more information.
Learn The Examples Of Sql Datediff()
Introduction to SQL Datediff()
In SQL server suppose we have dates in our data and we want to know the difference between those dates then we can use the DATEDIFF function to know the difference between those dates in days, months, or years. So this function returns an integer as output and to understand more about this function lets know it’s syntax first.
Start Your Free Data Science Course
Hadoop, Data Science, Statistics & others
Syntax of Datediff() in SQL DATEDIFF (interval, startdate, enddate)1. Interval – This is also called datepart and it is provided as a string to this function. This argument can be anything that represents a time interval like a month, week, day, year. We can also specify the quarter of the year.
year, yyyy, yy = Year SELECT DATEDIFF(year, '2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); quarter, qq, q = Quarter SELECT DATEDIFF(quarter,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); month, mm, m = month SELECT DATEDIFF(month, '2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); dayofyear = Day of the year SELECT DATEDIFF(dayofyear,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); day, dy, y = Da SELECT DATEDIFF(day,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); week, ww, wk = Week SELECT DATEDIFF(week,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); hour, hh = hour SELECT DATEDIFF(hour,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); minute, mi, n = Minute SELECT DATEDIFF(minute,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); second, ss, s = Second SELECT DATEDIFF(second,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); millisecond, ms = Millisecond SELECT DATEDIFF(millisecond,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000'); microsecond, mcs = Microsecond SELECT DATEDIFF(microsecond,'2010-12-31 23:59:59.9999999', '2011-01-01 00:00:00.0000000');2. startdate, enddate – These are the actual dates to get the difference between. This is a mandatory parameter.
This function works in the SQL server starting from the 2008 version, Azure SQL Data Warehouse, Azure SQL Database, Parallel Data Warehouse.
Return Value
The return value is an int and is expressed by the datepart or the interval boundary which is the difference between the start and end date.
If the range of the return value for int is out of[-2,147,483,648 to +2,147,483,647], DATEDIFF function returns an error. The max difference between the start and end date is 24 days, 20 hours, 31 minutes, and 23.647 seconds for the millisecond. The max difference is 68 years, 19 days, 3 hours, 14 minutes, and 7 seconds for the second.
If the start and end date have a date with different data type then DATEDIFF will set 0 the missing parts of the other date which has lower precision
The above queries have the same start and end values. These are adjacent dates and the difference between them is a hundred nanoseconds (.0000001 second). The start and end dates cross one calendar and the result of each query is 1.
ExamplesHere are the examples mention below
Example #1 – Calculating Age select ID,emp_name,emp_dateOfBirth from EmployeeWe have the above table Employee which consists of the date of birth and from this, we will calculate the age in terms of a year, month, and days in 2 steps
Step 1: Creating a function
CREATE FUNCTION fnEmpComputeAge(@EmpDOB DATETIME) RETURNS NVARCHAR(50) AS BEGIN DECLARE @AgeTempdate DATETIME, @AgeYears INT, @AgeMonths INT, @AgeDays INT SELECT @AgeTempdate= @EmpDOB SELECT @AgeTempdate=DATEADD(YEAR, @AgeYears, @AgeTempdate) SELECT @AgeTempdate=DATEADD(MONTH, @AgeMonths, @AgeTempdate) SELECT @AgeDays=DATEDIFF(DAY, @AgeTempdate,GETDATE()) DECLARE @EmpAge NVARCHAR(50) SET @EmpAge=Cast(@AgeYears AS NVARCHAR(4))+' AgeYears '+Cast(@AgeMonths AS NVARCHAR(2))+' AgeMonths '+Cast(@AgeDays AS NVARCHAR(2))+' AgeDays Old' RETURN @EmpAge EndIn the above example, we have created a SQL Function to calculate the age of the employee from the DOB so the function takes @EmpDOBas a parameter and returns NVARCHAR(50). We will see this in action when we run this function. In step, we have created this function.
Then we add the calculated years in the @AgeTempdate using the DATEADD function.
Similarly, we calculated the month and added in @AgeTempdate, and then it is used to calculate days. Next, we declared @EmpAge and set it to the concatenation of the final output. Since the calculation result is in int we used Cast function to convert it into nvarchar.
Step 2: Using the function in the query
select ID,emp_name,emp_dateOfBirth,dbo.fnEmpComputeAge(emp_dateOfBirth) as EmpAge from EmployeeThe result is as follows:
As we can see we have used dbo.fnEmpComputeAge function and passed emp_dateOfBirth to calculate EmpAge and the result is as above.
Example #2 – Using scalar functions and subqueries for start and end date SELECT DATEDIFF(day, (SELECT MIN([ShipDate])FROM Sales.SalesOrderHeader), (SELECT MAX([ShipDate])FROM Sales.SalesOrderHeader)) as ShippingDateDiff;The result is as follows:
In this example, we have calculated the shipping date difference using scalar functions and scalar subqueries for min and max.
Example #3 – Using ranking functions for the start date argument SELECT FirstName as first_name,LastName as last_name, DATEDIFF(day,ROW_NUMBER() OVER (ORDER BY DepartmentName),SYSDATETIME()) AS row_number FROM dbo.DimEmployee;The result is as follows:
In this function, we have used ROW_NUMBER() ranking function as the start date argument.
Example #4 – Using an aggregate window function for the start date argument SELECT FirstName as first_name,LastName as last_name,DepartmentName as department_name, DATEDIFF(year,MAX(HireDate) OVER (PARTITION BY DepartmentName),SYSDATETIME()) AS HireInterval FROM dbo.DimEmployee ConclusionHopefully, now you know what DATEDIFF() is in the SQL server and how it is used to calculate results the difference between date according to datepart.
Recommended ArticlesWe hope that this EDUCBA information on “SQL DATEDIFF()” was beneficial to you. You can view EDUCBA’s recommended articles for more information.
Different Types Of Frames In Html & Types Of Frame Tags
Introduction to Html Frames
Html frames are useful at dividing the web page or the web browser into multiple sections; they separate sections then load differently.
Start Your Free Software Development Course
Web development, programming languages, Software testing & others
A-frame displays content independent of its container. Multiple frames form a collection and are known as a frameset. The arrangement can be considered similar to the orientation of rows and columns in a table. The frame tag has been deprecated in HTML5.
Each frame has a frame tag to indicate. The horizontal frame is defined by the row attribute of the frame tag, and the vertical attribute is defined by the column attribute of the frame tag in the HTML document. The same window in one frame might display a static banner, a second navigation menu, etc.
Syntax of FramesSimilarly, as we saw above, the attribute rows take the value in pixels, and they help at specifying the number and size of rows in a frameset.
Different Type of Frames in HTMLThe different and some commonly used attributes in the frame are attributes such as border, scrolling, src, name, etc.
Types of Frame TagFollowing are the different types of frame tags:
1. Col: The col attribute gives the vertical frames. The width, however, can be specified in 4 ways:
Pixels: absolute values can be mentioned in pixels. If we have to create three vertical frames, we can give the value: cols=”100,50,100”.
Percentage: The percentage of the browser window can be mentioned. If we have to create three vertical frames, we can give the value: cols=”40%,20%,40%.”
We can also make use of the wildcard here (*) and let the wildcard take the remaining of the window, which remains un-mentioned.
cols=”30%,*,30%”
Output:
2. Rows: The row attribute gives the horizontal frames. It specifies the rows in a frameset. If we have to create three horizontal frames we use:
Eg: Rows=”10%,80%,10%’.
We can also set the height of each row as we had done in the case of columns previously.
Output:
3. Border: It specifies the width of the border of each frame in pixels.
E.g., border=”4”. If border=”0”, it means there is no border.
4. Frameborder: If a three-dimensional border needs to be displayed between frames, then we use this attribute. The value which the attribute takes is either 1 or 0 ( Yes or No).
E.g.: frameborder=”0” means no border.
5. Framespacing: This attribute specifies the amount of space between frames in a frameset. Any integer value can be given here for this attribute.
E.g., framespacing=”12” means between the frames, there should be the spacing of 12 pixels.
There are 8 attributes that can be listed here in this section.
1. src: We provide the file name to this attribute that is supposed to be loaded into the frame. The value of this attribute can be any URL.
E.g., name=”abc.htm.”
3. frameborder: This attribute is specified to show if the borders are to be shown or not. The value takes up is 1 or 0 (Yes or No).
4. marginwidth: The attribute is helpful at specifying the width of the space between the left and right of the frame’s border and also the content of the frame. The value is given in pixel.
5. marginheight: The attribute is helpful at specifying the height of the space between the top and the bottom of the frame’s border and also the content of the frame. The value is given in pixel.
E.g., marginheight=”10”.
6. noresize: This attribute basically prevents the user from making any changes to the already present frames. In the absence of this attribute, any frame can be resized.
E.g., noresize=”noresize.”
7. scrolling: This attribute takes control of the appearance of the scroll bars, which are present on the frame. The value either a ‘yes, ‘no’, or ‘auto’.
E.g., scrolling=” no” means it should not have scroll bars.
8. longdesc: This attribute lets you provide a link to another page containing a long description of the content of the frame.
E.g., longdesc=”framedesc.htm.”
Frames and Browser SupportIt is a nice approach to put up a message in such scenarios for the user who use old browsers. Messages like “ Sorry! Your browser does not support frames.” to notify.
ConclusionFrame technology is not supported by quite a number of browsers these days. It is often noticed that the contents of the web page are not displayed properly for smaller devices. Anyway, it is good to have knowledge of what existed so that better models could be developed to be compatible with new technologies.
Recommended ArticlesThis is a guide to Html Frames. Here we discuss the different types of Frames in HTML and the Types of Frame Tag with Frames and Browser Support. You may also look at the following article to learn more –
Update the detailed information about Learn The Types And Methods Of Mapreduce Api on the Achiashop.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!