' oicbasics: Read file using FTP adapter in OIC
August 14, 2019

Read file using FTP adapter in OIC

USE CASE : 


In this blog we will see how to READ a csv file records from FTP server directory. Inside this POC we will develop an integration which will take two INPUTS → File_Name, Directory_Path and will give file records as OUTPUT.

INPUT :


FTP .CSV FILE :

OUTPUT :






Lets see the development steps in detail:

STEP 1: In the Designer tab, click Integration and then click Create



      Select App Driven Orchestration



       Enter the below details and then click Create :

                   *What do you want to call your Integration? - write some meaningful name
                   *What can this Integration do? - write some meaningful description



STEP 2 : Search for the REST connection name ' RESTConnection ' and Select it.
                 ( You can access this blog to configure this REST connection )

       Enter the below details and click Next - [as shown in below image ]

       *What do you want to call your endpoint ? : write any meaningful name
              *What is the endpoint's relative resource URI ? :You can give any meaningful
                word with "/"
                Example :    /readFile
              *What action do you want to perform on the endpoint ? : Select POST
        &
        Select Configure a request payload for this endpoint
                Select Configure a response payload for this endpoint
  

   **Download & Save this below Schema as ReadFileRequest.xsd in your local system

     Download ReadFileRequest.xsd 

       Choose the above saved schema ReadFileRequest.xsd in next window and click 
       NEXT
      [as shown in below image ]


   **Save this below Schema as ReadFileResponse.xsd in your local system

        Download ReadFileResponse.xsd

       Choose the above saved schema ReadFileResponse.xsd in next window and click 
       NEXT

      [as shown in below image ]

     Click Done
      

STEP 3 : Search for the FTP connection name → " FTP_Test " which we have created in
                in the previous blog.



         Enter below details and then click NEXT :
                   *What do you want to call this endpoint : write any meaningful name
                   *What does this endpoint do : write meaningful description



         Enter the below details and then click NEXT :
                      *Select Operation : select Read a File
           *Select a Transfer Mode : select ASCII
            Input Directory : write /  (as shown in below image )
                        (here you can provide your directory path from where
             you want to read file)
                       File Name : Write EMP_1.csv
           (here you have to write file name which you want to read)



           Select Sample delimited document (e.g CSV ) and click Next 




         Paste the below text in notepad or any text editor tool and save it as 
        Employee.csv

Emp_Id,Emp_FirstName,Emp_LastName,Emp_Designation,Emp_DC_Location


            Configure the below details and then click NEXT :

                       *Choose File : select the above saved file Employee.csv
                       *Enter the Record Name:Employee (you can write any meaningful name)
                       *Enter the Recordset Name : EmployeeDetails
                         (you can write any meaningful name)

                    **Make sure to change the datatype of Emp_Id to Integer
                      (as shown in below image)

     Click Done 



STEP 4 : Open the FTP adapter mapper and map the columns accordingly 
                (as shown in below image )






STEP 5 : Now open the response mapper 
             
           and map the columns accordingly (as shown in below image )
           
          **Make sure to map the Employee node also to configure for each




       you can see the for each function by clicking on Advance tab



    click Validate and Close the mapper.

STEP 6 : Add the Tracker for your Integration :
              Right click on Hamburger sign and then click on Tracking


            Map File_Name as tracking variable and then click Save
             (as shown in below image )


STEP 7 : Click Save and then Close the integration. Your Integration is ready to test.





TESTING STEPS :


In below Step 1 we will put the file in FTP server directory (at / directory)which we want to read and in Step 2 we will RUN the integration using SoapUI tool.


STEP 1:


   Open FileZilla Tool :

   Enter the FTP Connection details (you can see the previous blog for FTP Credentials)


   Click OK

    

   Save the below .csv file as EMP_1.csv in your local system 

Emp_Id,Emp_FirstName,Emp_LastName,Emp_Designation,Emp_DC_Location
1001,Rohit,Kuma,System Engineer,Delhi
1002,Rahul,Kumar,System Engineer,Pune

or

Download EMP_1.csv file

            Upload the above EMP_1.csv file from your local system to FTP  server  
            directory (as shown in below image)



NOW....

STEP 2 :

    Activate the Integration 



      Click Activate


     Click on Endpoint URL  (as shown in below image )




     Copy the Endpoint URL 



     Open SoapUI tool (5.5.0):




  • Click on REST
  • Then Paste the above copied endpoint URL and then click OK



  • Configure the REQUEST PAYLOAD ( as shown in below image):
       Method : POST
       Media Type : application/xml
      Download application/xml payload

     paste the above downloaded SoapUI payload in application/xml payload (as shown in below
     image)




            Authorization : Add Basic autorization 



                                  Enter your OIC userid and password

  

     Now click on RUN button. You will get the read file Response 

      **Note you may see Socket Time out Error in SOAPUI with empty Response
          payload.  
          (This issue occurs  when the server takes more than one minute to respond)



    To resolve this issue,Open SoapUI -> Preferences ->HTTP Settings ->Update
      Socket Timeout(ms) to 600000 and then click Ok


   Now if you again hit the RUN button you will successfully able to see the response.




Thank You !

6 comments:

  1. Thanks too much.
    Your documnetation very clear.

    I'm asking if it's posssible to read other files in specific directory.

    ReplyDelete
    Replies
    1. you are welcome :)

      You mean other file formats ? yes OIC flow can read

      Delete
  2. Thank you Kabir for the blog. Could you please tell me how to handle large files in OIC.

    ReplyDelete
    Replies
    1. Welcome Giri :)

      Yes you can read large files in OIC , use FTP Adapter operation 'Read File in Segment' inspite of 'Read File'.

      Delete
    2. In above article Step -3 , Select Operation - there you select 'Read file in Segment '

      Delete
    3. When file size is up to 1GB then we can FTP 'download' and stage 'read in segments'. But if file size is more than 1GB for example 1.5GB can u please tell me how to read this file from FTP folder using OIC?

      Delete

If you have any doubts, Please let me know.

Top