Notices: This section not yet converted to new layout. Download stats are rolling back out.

Hadoop

3.1.1

Package test results are passing.

Private CDN cached downloads available for licensed customers. Never experience 404 breakages again! Learn more...

This package was approved as a trusted package on 8/10/2018.

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.

The project includes these modules:

  • Hadoop Common: The common utilities that support the other Hadoop modules.
  • Hadoop Distributed File System (HDFS™): A distributed file system that provides high-throughput access to application data.
  • Hadoop YARN: A framework for job scheduling and cluster resource management.
  • Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.

Notes

The package sets the HADOOP_HOME environment variable and adds HADOOP_HOME\bin to the machine path.

The package by default will unzip to C:\Hadoop. This location can be changed, see below.

Package Parameters

The following package parameters can be set:

  • /unzipLocation - Unzip to a different path than default.

These parameters can be passed to the installer with the use of -params.
For example: choco install hadoop -params '"/unzipLocation:D:\Hadoop"'.

To install Hadoop, run the following command from the command line or from PowerShell:

C:\> choco install hadoop

To upgrade Hadoop, run the following command from the command line or from PowerShell:

C:\> choco upgrade hadoop

Files

Hide
  • tools\chocolateyinstall.ps1 Show
    $ErrorActionPreference = 'Stop';
    
    $packageName    = 'hadoop'
    $packageVersion = $env:chocolateyPackageVersion
    $toolsDir       = "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"
    $hadoop_home    = "C:\Hadoop"
    $mirrors        = 'http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.1/hadoop-3.1.1.tar.gz'
    $checksum32     = 'f837fe260587f71629aad1f4fb6719274e948111dc96ffc5a8e26f27deac5602'
    
    # d/l from closest mirror
    $get_mirror_page = Invoke-WebRequest -Uri $mirrors -UseBasicParsing
    $url32 = $get_mirror_page.links | ? href -match '\.tar\.gz$' | select -First 1 -expand href
    
    
    $pp = Get-PackageParameters
    if ($pp.unzipLocation) { 
        $hadoop_home = $pp.unzipLocation
        Write-Host "Param: Unzipping (installing) to $hadoop_home"
    }
    
    
    $packageArgs = @{
      packageName   = $packageName
      unzipLocation = $toolsDir
      url           = $url32
      checksum      = $checksum32
      checksumType  = 'sha256'
      validExitCodes= @(0)
    }
    
    Install-ChocolateyZipPackage @packageArgs
    
    # unzip tar
    Get-ChocolateyUnzip -FileFullPath "$toolsDir\*.tar" -Destination $hadoop_home
    
    Install-ChocolateyEnvironmentVariable `
        -VariableName "HADOOP_HOME" `
        -VariableValue $hadoop_home `
        -VariableType 'Machine'
    
    Install-ChocolateyPath `
        -PathToInstall "%HADOOP_HOME%\$packageName-$packageVersion\bin" `
        -PathType 'Machine'
    
    # Hadoop needs 8.3 path to find Java    
    $sp = New-Object -ComObject Scripting.FileSystemObject
    $f = $sp.GetFolder($env:JAVA_HOME)
    
    Install-ChocolateyEnvironmentVariable `
        -VariableName "JAVA_HOME" `
        -VariableValue $f.ShortPath `
        -VariableType 'User'
    
    # don't need tar anymore
    Remove-Item $toolsDir\*.tar -ErrorAction SilentlyContinue -Force
    
  • tools\chocolateyuninstall.ps1 Show
    $packageName = 'hadoop'
    $packageVersion = $env:chocolateyPackageVersion
    $hadoop_home = $env:HADOOP_HOME
    
    # remove from Path
    $path = [Environment]::GetEnvironmentVariable("Path", 'Machine')
    $newPath = ($path.Split(';') | Where-Object { $_ -notmatch '.*hadoop.*' }) -join ';'
    [Environment]::SetEnvironmentVariable("Path", $newPath, 'Machine')
    
    
    Install-ChocolateyEnvironmentVariable `
        -VariableName "HADOOP_HOME" `
        -VariableValue $null `
        -VariableType 'Machine'
    
    Install-ChocolateyEnvironmentVariable `
        -VariableName "JAVA_HOME" `
        -VariableValue $null `
        -VariableType 'User'
    
    
    Remove-Item $hadoop_home -Recurse -Force
    

Virus Scan Results

In cases where actual malware is found, the packages are subject to removal. Software sometimes has false positives. Moderators do not necessarily validate the safety of the underlying software, only that a package retrieves software from the official distribution point and/or validate embedded software against official distribution point (where distribution rights allow redistribution).

Chocolatey Pro provides runtime protection from possible malware.

Dependencies

Package Maintainer(s)

Software Author(s)

  • Apache Software Foundation

Copyright

Copyright © 2014 The Apache Software Foundation

Tags

Release Notes

http://hadoop.apache.org/releases.html

Version History

Version Downloads Last updated Status
Hadoop 3.1.0 237 Monday, April 9, 2018 approved
Hadoop 3.0.1 81 Tuesday, March 27, 2018 approved
Hadoop 3.0.0 235 Friday, December 15, 2017 approved
Hadoop 2.9.0 162 Tuesday, November 21, 2017 approved
Hadoop 2.8.2 160 Monday, November 6, 2017 approved
Hadoop 2.8.1 282 Monday, July 24, 2017 approved
Hadoop 2.8.0 280 Saturday, April 1, 2017 approved
Hadoop 2.7.3 194 Monday, February 27, 2017 approved
Show More

Discussion for the Hadoop Package

Ground rules:

  • This discussion is only about Hadoop and the Hadoop package. If you have feedback for Chocolatey, please contact the google group.
  • This discussion will carry over multiple versions. If you have a comment about a particular version, please note that in your comments.
  • The maintainers of this Chocolatey Package will be notified about new comments that are posted to this Disqus thread, however, it is NOT a guarantee that you will get a response. If you do not hear back from the maintainers after posting a message below, please follow up by using the link on the left side of this page or follow this link to contact maintainers. If you still hear nothing back, please follow the package triage process.
  • Tell us what you love about the package or Hadoop, or tell us what needs improvement.
  • Share your experiences with the package, or extra configuration or gotchas that you've found.
  • If you use a url, the comment will be flagged for moderation until you've been whitelisted. Disqus moderated comments are approved on a weekly schedule if not sooner. It could take between 1-5 days for your comment to show up.

comments powered by Disqus
Chocolatey.org uses cookies to enhance the user experience of the site.
Ok