 HORTONWORKS.

COM Explore  Create  Tracks  Entrar

HCC Hortonworks Community Connection Encontrar publicações, tópicos e utilizadores... 

Inicio / Hadoop Core /

[Closed] : How to store output of shell script in HDFS

Pergunta por JAy PaTel 15 de Jun de 2018 às 11:07 H D F S hdfs-permissions script

HI All,
I have a shell script on HDFS as well as the locally named script.sh contains echo Hi.
I could execute the script.sh file on locally and store output on locally of course.
But I want to execute script.sh file (wherever on local or on HDFS) and store output on HDFS.
I have done following;
script.sh
#!/bin/bash
echo "`date` hi" > /tmp/output

bash script.sh
above command ran successfully. but if I changed the output path it is giving me an error that ;
script.sh: line 2: hdfs://<host>:<port>/user/oozie/output/shell: No such file or directory 81
SEGUIDORES
#!/bin/bash
 Responstas
echo "`date` hi" > hdfs://<HOST>:<PORT>/user/oozie/output/shell

 Respostas E
Comentários

Kindly help me for this.

Thank you,
Jay.
 shell.jpg (40.8 kB)
0

2Respostas · Adicionar a sua resposta

MELHOR RESPOSTA

Resposta por Aditya Sirna  · 15 de Jun de 2018 às 11:58

@JAy PaTel,

You cannot directly write the output of echo to hdfs file. Instead you can do like below
echo "`date` hi" > /tmp/output ; hdfs dfs ‐appendToFile /tmp/output /tmp/abc.txt

.
-Aditya

 1  Esconder 1 · Partilhar

JAy PaTel · 16 de Jun de 2018 às 06:28 0

@Aditya Sirna

Thank you Aditya. Your observation worked for me.

NOTE:

echo "`date` hi">/tmp/output ; hdfs dfs ‐appendToFile <local_directory_path> <hdfs_directory_path>

Regards,
Jay.

Resposta por Kevin Whitson · 27 de Dec de 2018 às 04:04

This can be done without first writing to local disk...

echo "`date` hi" | hdfs dfs ‐appendToFile ‐ /tmp/abc.txt

The hyphen "-" reads from stdin

 3  Esconder 1 · Partilhar

Manish Singh · 07 de Feb às 06:18 0

Awesome!!!

Sua resposta
HTML Editor

Preview Source

                  

Preencha os detalhes...
0
Dica: Pode notificar um utilizador sobre esta publicação digitando @username

Não sou um robô

reCAPTCHA
Privacidade - Termos

PUBLICAR RESPOSTA
THIS WEBSITE USES COOKIES FOR ANALYTICS, PERSONALISATION AND ADVERTISING. TO LEARN MORE OR CHANGE YOUR COOKIE SETTINGS, PLEASE READ OUR COOKIE POLICY. BY CONTINUING TO
BROWSE, YOU AGREE TO OUR USE OF COOKIES.