Azure file copy creating a directory to hold the renamed file

• Irish-Toon

  SSC Enthusiast

  Points: 174

  More actions

  November 14, 2019 at 8:49 am

  #3697584

  Hi All,

  I'm pretty new to azure and am hitting the following problem. I've got a notebook that creates an output csv file. The file gets named to the default Part-0000-tid and I want to rename it to something more sensible. The code below shows what I've done. The problem is it creates a subfolder in the destination folder with the new filename and then within that folder it has the renamed file ( see attached screenshot). It looks like I'm close but no quiet right. I need the new file to be up 1 level and appear in the outbound folder. Any help would be appreciated

  %python

  import os, sys, datetime

  readPath = "/mnt/publisheddatasmets1mig/metering/smets1mig/cs/system_data_build/notes/outbound/"

  writePath = "/mnt/publisheddatasmets1mig/metering/smets1mig/cs/system_data_build/notes/outbound/"

  file_list = dbutils.fs.ls(readPath)

  for i in file_list:

  file_path = i[0]

  file_name = i[1]

  file_name

  Current_Date = datetime.datetime.today().strftime ('%Y-%m-%d-%H%M%S')

  fname = "CS_Notes_" + str(Current_Date) + ".csv"

  for i in file_list:

  if i[1].startswith("part-00000"):

  dbutils.fs.cp(readPath+file_name,writePath+fname)

   

  Attachments:

  You must be logged in to view attached files.

• Site Owners

  SSC Guru

  Points: 80330

  More actions

  November 15, 2019 at 9:10 am

  #3697962

  Thanks for posting your issue and hopefully someone will answer soon.

  This is an automated bump to increase visibility of your question.