August 4, 2017 at 3:19 am
Hi,
I'm having trouble with external pushdown for polybase when connecting to my hadoop (Cloudera) Cluster. When running a select statement, I receive the following error:
SELECT * FROM [dbo].[CarSensor_Data]OPTION (FORCE EXTERNALPUSHDOWN);
Cannot execute the query "Remote Query" against OLE DB provider "SQLNCLI11" for linked server "(null)". EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to JobSubmitter_SubmitJob: Error [Requested replication factor of 1 is less than the required minimum of 2 for /user/waltonc/.staging/job_1499778383800_13648/job.jar from"
I have dfs.replication.min set to 2, so I understand that HDFS will now allow the job.jar file to be created on it's own and that it must have a replication of at least two, however i'm unable to find where this setting needs to be.
I've tried adding into the yarn-site.xml, mapred-site.xml, hdfs-site.xml and core-site.xml, but no luck.
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
Any ideas?
October 31, 2017 at 12:13 am
Hi Chris,
The permissions or user id having access to Hadoop is struggling to create its home directory in HDFS. You could try adding the permissions manually to Hadoop.
Regards,
Kanika Soni
Viewing 2 posts - 1 through 1 (of 1 total)
You must be logged in to reply to this topic. Login to reply