Hadoop’s org.apache.hadoop.fs.FileSystem is generic class to access and manage HDFS files/directories located in distributed environment. File’s content stored inside datanode with multiple equal large sizes of blocks (e.g. 64 MB), and namenode keep the information of those blocks and Meta information. FileSystem read and stream by accessing blocks in sequence order. FileSystem first get blocks information from NameNode then open, read and close one by one. It opens first blocks once it complete then close and open next block. HDFS replicate the block to give higher reliability and scalability and if client is one of the datanode then it tries to access block locally if fail then move to other cluster datanode. Continue reading “Hadoop HDFS JAVA API”
Category: Tutorial
Hadoop MapReduce
Introduction
HDFS stores file in multiple equal large size block e.g. 64 MB, 128 MB etc. and MapReduce framework access and process these files in distributed environment.
The MapReduce framework works on key-value pairs, it has two key part Mapper and Reducer.Map Reducers read file and split and pass to Mapper. Mapper set the input as key-value pairs and pass to the intermediate for sorting and shuffling. Reducer takes the key and list of value, process and writes to the disk. Continue reading “Hadoop MapReduce”
Apache Hadoop Setup
Hadoop 2.x is based on YARN architecture, which uses ResourceManagaer and ApplicationManager. ResourceManagaer manage recourses across cluster and Application Manager manages job life cycles. Continue reading “Apache Hadoop Setup”
Grails UnitTesting & Deployment
Unit Testing
GrailsUnitTestCase and GroovyTestCase both extend JUnit 3.x TestCase.MVC Unit Test case will extend GrailsUNitTest case and delegate the test case control to ControllerUnitTestCase and TagLibUnitTestCase
Grails Services
Services
Grails Service uses to separate your business logic from the other layer or component. Services are named in the format SomeNameService.groovy and placed in the directory /grails-app/services/. Services can make use of dependency injection features, and you can easily call these services from within controllers. Continue reading “Grails Services”
Grails – GSP
GSP
Grails uses GSP for the presentation tier. The Groovy in Groovy Server Pages not only identifies the underlying technology, but also the language you can use if you want to write a quick scriptlet or two. Groovy Taglib and partial templates give you a more sophisticated way of sharing code and behavior across Web pages. Continue reading “Grails – GSP”
Grails Scaffolding Controllers and Views
Controllers and views round out the “Big Three” in Grails applications. The Model/View/Controller (MVC) pattern is undoubtedly familiar to most web developers. By enforcing a clean separation of concerns among these three elements, your efforts are paid back in terms of maximum flexibility and reuse. Continue reading “Grails Scaffolding Controllers and Views”
TechMyTalk 