Building Hadoop-based Apps on YARN
来源:互联网 发布:linux双网关配置 编辑:程序博客网 时间:2024/05/01 02:12
Apache Hadoop YARN changes the game for Hadoop applications, enabling a multi-application, multi-workload general purpose data operating system. YARN is:
- Flexible
Store data once and interact with it in multiple ways from batch to interactive to real time and streaming.
Architected to enable new workloads.
- Shared
Re-use key platform services for reliability, redundancy and security across multiple workloads.
Multi-tenant architecture shares core resources while isolating services and data.
- Efficient
Do more with less: 30%+ increased efficiency on existing resource utilization.
Share and segment applications based on cluster resource management.
This set of resources is intended to get you up and running developing apps for YARN.
STEP 1. Understand the motivations and architecture for YARN.
Apache Hadoop YARN is the data operating system for Hadoop 2.0. YARN enables a user to interact with all data in multiple ways simultaneously, making Hadoop a true multi-use data platform and allowing it to take its place in a modern data architecture. Find out more about the concepts and specifics of YARN.
Get an overview of Apache Hadoop YARN concepts in this slide deck.
Concepts
- Introducing Apache Hadoop YARN
- Apache Hadoop YARN – Background and an Overview
- Apache Hadoop YARN – Concepts and Applications
- Apache Hadoop YARN – ResourceManager
- Apache Hadoop YARN – NodeManager
Building Apps
- Running existing applications on Hadoop 2 YARN
- Stabilizing YARN APIs for Apache Hadoop 2
- Management of Application Dependencies
- Resource Localization in YARN: Deep Dive
- Simplifying user-logs management and access in YARN
STEP 2. Explore example applications on YARN.
The simple applications in this section show how to build and deploy apps against the YARN APIs and are a simple way to get started. These apps can be easily replicated in the Hortonworks Sandbox VM environment.
- Simple YARN App. This ‘Hello World’ app for YARN runs n copies of a unix command.
- Distributed Shell. This fuller example implements a distributed shell on YARN.
- MemcacheD on YARN. A tutorial showing how to deploy the very popular MemcacheD framework on YARN.
STEP 3. Examine real world applications YARN.
These applications are richer applications built on YARN and demonstrate real-world use and deployment.
- MapReduce on YARN The official codebase for Apache Hadoop MapReduce on YARN (MR2)
- HBase on YARN. Efforts to deploy HBase on YARN.
FURTHER RESOURCES
The following resources can also assist with developing Hadoop-based Apps on YARN.
- Apache Hadoop YARN – Enabling Next Generation ApplicationsGet Started with Hadoop 2.0 with this reference presentation
- Sample eBook Chapters – Apache Hadoop YARN
- Find more blog posts related to Apache Hadoop YARN
TRAINING
Hortonworks also provides training and certification for Hadoop.
- Hadoop Essentials 1-Day Class
- Hadoop Training for Java Developers – 4-Day Class
- Hadoop on Windows – 4-Day Class
- Hadoop for Data Analysts – 4-Day Class
- Building Hadoop-based Apps on YARN
- hadoop on yarn and spark on yarn
- Building your own kernel based on CentOS
- 搭建Hadoop Yarn on Docker
- Spark on Hadoop Yarn 部署
- Spark on yarn和Hadoop on yarn的区别
- Java Based Apps on SAP Cloud_0_系统准备
- Java Based Apps on SAP Cloud_1_总体介绍
- Java Based Apps on SAP Cloud_2_开发ESPM
- Building GTK apps for MS Windows on Linux
- Hadoop YARN新特性—label based scheduling
- Hadoop YARN新特性—label based scheduling
- Hadoop YARN新特性—label based scheduling
- spring-hadoop wordcount run on yarn
- Hadoop on YARN参数配置讲解
- Building online HBase cluster of Zhihu based on Kubernetes
- hadoop解决Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/util/Apps
- Building Apps with Multimedia
- Live555 RTSP RTP库思考
- unity优化DrawCall
- UML(六)活动图
- Modern Manufacturing Architectures Built with Hadoop
- java二维数组遍历
- Building Hadoop-based Apps on YARN
- ubuntu解压命令全览
- OpenGL学习(六)锯齿
- Lightoj 1030 概率DP
- <sys/stat.h>的含义!
- NotificationManager和Notification的使用总结
- 接口Cloneable浅复制和深复制
- Listview的下拉刷新
- asp.net页面中实现如果图片不存在则显示默认图片