Spring Boot Read a File From Desktop

Spring Tips: Remote File System Integrations (FTP) with Spring Integration

Spring Tips: FTP Integration

speaker: Josh Long (@starbuxman)

Howdy, Leap fans! In this installment of Spring Tips, we look at a topic that's well-nigh and honey to my heart: integration! And yeah, you lot may recall that the very beginning installment of Spring Tips looked at Spring Integration. If you haven't already watched that one, you should. So, while nosotros're not going to revisit Spring Integration fundamentals, we're going to take a deep dive into 1 area fo support in Spring Integration: FTP. FTP is all about file synchronization. Broadly, in the globe of Enterprise Application Integration (EAI), nosotros have four types of integration: file synchronization, RPC, database synchronization, and messaging.

File synchronization is definitely non what most people think of when they recall of cloud-native applications, only yous'd exist surprised just how much of the world of finance is run past file synchronization (FTP, SFTP, AS2, FTPS, NFS, SMB, etc.) integrations. Certain, almost of them utilise the more secure variants, merely the point is notwithstanding valid. In this video, nosotros wait at how to apply Bound Integration's FTP support, and one time you understand that, it's easy enough to apply it to other variants.

Please indulge me in a chip of chest-thumping hither: I thought that I knew everything I'd needed to know about Spring Integration'due south FTP support since I had a major function in polishing off Iwein Fuld's original image code more than a decade ago, and since I contributed the original FTPS and SFTP adapters. In the intervening decade, surprising nobody, the Jump Integration team has added a ton of new capabilities and stock-still all the bugs in my original code! I love what's been introduced.

And so, showtime things first: nosotros need to ready up an FTP server. Nearly of Bound Integration's support works every bit a client to an already installed FTP server. And so, it doesn't matter what FTP server you use. Withal, I'd recommend you utilize the Apache FTPServer project. It's a project that'south a sub-projection of the Apache Mina project, which is, only and then you lot know, the precursor to the Netty project. The Apache FTP Server is a super scalable, lightweight, all-Java implementation of the FTP protocol. And, y'all can easily embed it within a Spring application. I've done and then in the Github repository for this video. I divers a custom UserManager class to manage FTP user accounts. The custom UserManager that talks to a local PostgreSQL database with a elementary table ftp_user, whose schema is defined as a table with the post-obit columns:

I've got two users in there, jlong and grussell, both of which have a password of pw. I've prepare enabled and admin to truthful for both records. We utilise these two accounts later, so make sure yous insert them into the tabular array, similar this.

                insert into ftp_user(username, password, enabled, admin) values ('jlong', 'pw', truthful, true); insert into ftp_user(username, countersign, enabled, admin) values ('grussell', 'pow', true, truthful);                              

I'k non going to reprint the code for the FTP server here in its entirety. If you desire to peruse it, I'd recommend yous look at the FtpServerConfiguration and FtpUserManager.

In most cases, we don't take any ability to change the FTP server. If nosotros want to be notified of any changes in a remote file system, our client needs to connect, scan the directory, and compare it with an earlier, known state. Basically, the client computes the delta and publishes an consequence. Simply wouldn't information technology be nice if the FTP server could broadcast an event when something happens? That mode, there can be no doubt most what happened. And there's no doubt that we observed every change. If we were using any other FTP server, this would exist more of a wish than a possibility. Just equally we're using the Apache FTP Server, Spring Integration offers us some interesting possibilities. Nosotros can install an FTPlet, kind of like a filter, that volition circulate whatsoever of import events on the FTP server as ApplicationContext events. Then, we can use Spring Integration to publish interesting events as letters that we tin can process in Spring Integration. This capability is a new feature in Spring Integration.

                packet ftp;  import lombok.extern.log4j.Log4j2; import org.springframework.context.note.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.integration.dsl.IntegrationFlow; import org.springframework.integration.dsl.IntegrationFlows; import org.springframework.integration.dsl.MessageChannels; import org.springframework.integration.effect.inbound.ApplicationEventListeningMessageProducer; import org.springframework.integration.ftp.server.ApacheMinaFtpEvent; import org.springframework.integration.ftp.server.ApacheMinaFtplet; import org.springframework.integration.handler.GenericHandler; import org.springframework.messaging.MessageChannel;  @Log4j2 @Configuration class IntegrationConfiguration {  	@Bean 	ApacheMinaFtplet apacheMinaFtplet() { 		return new ApacheMinaFtplet(); 	}  	@Bean 	MessageChannel eventsChannel() { 		render MessageChannels.direct().become(); 	}  	@Bean 	IntegrationFlow integrationFlow() { 		return IntegrationFlows.from(this.eventsChannel()) 			.handle((GenericHandler<ApacheMinaFtpEvent>) (apacheMinaFtpEvent, messageHeaders) -> {                 log.info("new event: " + apacheMinaFtpEvent.getClass().getName() +                       ':' + apacheMinaFtpEvent.getSession()); 				return null; 			}) 			.go(); 	}  	@Bean 	ApplicationEventListeningMessageProducer applicationEventListeningMessageProducer() { 		var producer = new ApplicationEventListeningMessageProducer(); 		producer.setEventTypes(ApacheMinaFtpEvent.class); 		producer.setOutputChannel(eventsChannel()); 		return producer; 	} }                              

This example sets upwards a Spring Integration messaging flow that listens for the relevant events and logs them out. Plainly, nosotros're doing too much with this new information, but the affair to proceed in heed is that… we TOTALLY could! There are so many opportunities here. We could publish the events over Apache Kafka, RabbitMQ, or JMS for another node to respond to. We could send an email inviting someone to participate in some workflow. The sky's the limit!

Now, we've got a working server up and running on port 7777, we can connect using a client. I use Filezilla. Whatsoever client you use, endeavour logging into the running FTP server on host localhost, port 7777, user jlong, and password pw. Upload a file, rename it, etc., and and then check the panel of your application and you lot'll encounter the action reflected in events.

The FTP Client

We've got a working server. Let's look at how Spring Integration tin act equally a client to your services. Nosotros'll work with the simplest abstraction and piece of work our style upward to more sophisticated capabilities. Create a new project on the Bound Initializr, add Lombok, Jump Integration, and choose the latest version of Java. Then click Generate and open the projection in your IDE.

Nosotros're going to utilise the two accounts nosotros divers earlier. Let's configure them in the awarding.properties.

                ## ## Josh ftp1.username=jlong ftp1.password=pw ftp1.port=7777 ftp1.host=localhost ##  ## Gary ftp2.username=grussell ftp2.password=pw ftp2.port=7777 ftp2.host=localhost                              

The FtpRemoteFileTemplate

The simplest mode nosotros can collaborate with an FTP server is to employ the very handy FtpRemoteFileTemplate that ships as part of Spring Integration. Here's an example. This commencement example defines a DefaultFtpSessionFactory that establishes a connection to one of the FTP accounts. And then we define a FtpRemoteFileTemplate using that DefaultFtpSessionFactory. Then, we define an initializer that uses that FtpRemoteFileTemplate to read a file on the remote file system, hello.txt, to a local file, $HOME/Desktop/hi-local.txt. Information technology couldn't be simpler!

                package com.example.integration;  import lombok.extern.log4j.Log4j2; import org.springframework.beans.factory.InitializingBean; import org.springframework.beans.factory.notation.Value; import org.springframework.context.note.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Profile; import org.springframework.integration.ftp.session.DefaultFtpSessionFactory; import org.springframework.integration.ftp.session.FtpRemoteFileTemplate;  import java.io.File; import java.io.FileOutputStream;  @Log4j2 @Configuration class FtpTemplateConfiguration {  	@Bean 	InitializingBean initializingBean(FtpRemoteFileTemplate template) { 		render () -> template 			.execute(session -> { 				var file = new File(new File(Organization.getProperty("user.habitation"), "Desktop"), "hello-local.txt"); 				try (var fout = new FileOutputStream(file)) { 					session.read("how-do-you-do.txt", fout); 				} 				log.info("read " + file.getAbsolutePath()); 				return null; 			}); 	}  	@Edible bean 	DefaultFtpSessionFactory defaultFtpSessionFactory( 		@Value("${ftp1.username}") String username, 		@Value("${ftp1.password}") String prisoner of war, 		@Value("${ftp1.host}") Cord host, 		@Value("${ftp1.port}") int port) { 		DefaultFtpSessionFactory defaultFtpSessionFactory = new DefaultFtpSessionFactory(); 		defaultFtpSessionFactory.setPassword(pw); 		defaultFtpSessionFactory.setUsername(username); 		defaultFtpSessionFactory.setHost(host); 		defaultFtpSessionFactory.setPort(port); 		render defaultFtpSessionFactory; 	}  	@Bean 	FtpRemoteFileTemplate ftpRemoteFileTemplate(DefaultFtpSessionFactory dsf) { 		return new FtpRemoteFileTemplate(dsf); 	} }                              

The FTP Entering Adapter

The next example looks at how to utilize an FTP inbound adapter to receive a new Message<File> whenever at that place's a new file on the remote file system. An inbound or outbound adapter is a unidirectional messaging component. An inbound adapter translates events from a remote system into new messages that are delivered into a Bound Integration period. An outbound adapter translates a Spring Integration Bulletin<T> into an event in a remote system. In this case, the FTP inbound adapter will publish a Message<T> into the Spring Integration code whenever a new file appears on the remote file system.

As before, we configure a DefaultFtpSessionFactory. Then, nosotros configure an FTP inbound adapter that automatically synchronizes the remote file organisation whenever whatsoever file that matches the mask .txt arrives on the server. The inbound adapter takes the remote file, moves information technology to the local directory, then publishes a Bulletin<File> that we tin can do annihilation we'd like with. Here, I merely log the message. Try it out! Upload a file, foo.txt, to the FTP server and watch equally - no more than a second later - information technology'due south downloaded and stored in the local file system nether $Home/Desktop/local.

                                  
packet com.instance.integration;  import lombok.extern.log4j.Log4j2; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.notation.Profile; import org.springframework.integration.dsl.IntegrationFlow; import org.springframework.integration.dsl.IntegrationFlows; import org.springframework.integration.ftp.dsl.Ftp; import org.springframework.integration.ftp.session.DefaultFtpSessionFactory;  import java.io.File; import java.util.concurrent.TimeUnit;  @Log4j2 @Configuration class InboundConfiguration {  	@Bean 	DefaultFtpSessionFactory defaultFtpSessionFactory( 		@Value("${ftp1.username}") String username, 		@Value("${ftp1.password}") String pw, 		@Value("${ftp1.host}") Cord host, 		@Value("${ftp1.port}") int port) { 		DefaultFtpSessionFactory defaultFtpSessionFactory = new DefaultFtpSessionFactory(); 		defaultFtpSessionFactory.setPassword(pw); 		defaultFtpSessionFactory.setUsername(username); 		defaultFtpSessionFactory.setHost(host); 		defaultFtpSessionFactory.setPort(port); 		return defaultFtpSessionFactory; 	}  	@Bean 	IntegrationFlow inbound(DefaultFtpSessionFactory ftpSf) { 		var localDirectory = new File(new File(System.getProperty("user.dwelling"), "Desktop"), "local"); 		var spec = Ftp 			.inboundAdapter(ftpSf) 			.autoCreateLocalDirectory(true) 			.patternFilter("*.txt") 			.localDirectory(localDirectory); 		render IntegrationFlows 			.from(spec, pc -> pc.poller(pm -> pm.fixedRate(1000, TimeUnit.MILLISECONDS))) 			.handle((file, messageHeaders) -> { 				log.info("new file: " + file + "."); 				messageHeaders.forEach((yard, v) -> log.info(g + ':' + v)); 				return null; 			}) 			.get(); 	} }                              

The FTP Gateway

Now, for our concluding cease, let'southward look at the Spring Integration FTP Gateway. In Spring Integration, a gateway is a component that sends data out (to a remote service) and so takes the response and brings it back into the Spring Integration flow. Or, alternatively, a gateway could have an incoming request from a remote organization, bring it into the Spring Integration flow, and then ship a response back out again. Either manner, a gateway is a bidirectional messaging component. In this instance, the FTP gateway takes Spring Integration Message<T>southward, sends them to an FTP server and uploads them, and once they're uploaded, ship the response (the acknowledgment, if nothing else) dorsum into the Bound Integration code.

That would be useful in of itself if that's all we did. Simply, for this last example, I want to conditionally upload a file to one of two FTP server accounts based on some criteria. You can imagine the scenario. An HTTP request comes, it's turned into a Message<T> that enters the Spring Integration menses, and it heads to the gateway. The merely question is: to which account should the information be uploaded? Jane would probably not appreciate it if a file intended for John was uploaded to her account.

We're going to use a DelegatingSessionFactory<FTPFile>. The DelegatingSessionFactory<FTPFile> has two constructors. One takes a SessionFactoryLocator, which you can use to brand the decision at runtime which FTP account to apply. The other takes a Map<String, SessionFactory> which in turn results in a SessionFactoryLocator that looks at some property of an incoming bulletin (it's up to you lot which) and uses that equally the key for a lookup in the map.

We need some way to kick off the pipeline, so I created a simple HTTP endpoint that accepts an HTTP Postal service message and uses a path variable to establish a key that and then gets sent into the integration flow. The integration flow has iii steps. The beginning stage looks at the incoming bulletin and configures the thread-local key for the DelegatingSessionFactory, then information technology forrad the message to the gateway which does the piece of work of uploading the file to a remote file organisation, then information technology forwards the response from the upload to another component which clears the thread-local fundamental.

                bundle com.instance.integration;  import org.apache.commons.net.ftp.FTPFile; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Profile; import org.springframework.expression.common.LiteralExpression; import org.springframework.integration.dsl.IntegrationFlow; import org.springframework.integration.dsl.MessageChannels; import org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway; import org.springframework.integration.file.remote.session.DelegatingSessionFactory; import org.springframework.integration.file.support.FileExistsMode; import org.springframework.integration.ftp.dsl.Ftp; import org.springframework.integration.ftp.session.DefaultFtpSessionFactory; import org.springframework.integration.ftp.session.FtpRemoteFileTemplate; import org.springframework.integration.handler.GenericHandler; import org.springframework.messaging.MessageChannel; import org.springframework.messaging.back up.MessageBuilder; import org.springframework.web.servlet.function.RouterFunction; import org.springframework.web.servlet.function.ServerResponse;  import java.util.Map;  import static org.springframework.web.servlet.role.RouterFunctions.route;  @Configuration @Profile("gateway") class GatewayConfiguration {  	@Bean 	MessageChannel incoming() { 		return MessageChannels.straight().get(); 	}  	@Edible bean 	IntegrationFlow gateway( 		FtpRemoteFileTemplate template, 		DelegatingSessionFactory<FTPFile> dsf) { 		return f -> f 			.channel(incoming()) 			.handle((GenericHandler<Object>) (central, messageHeaders) -> { 				dsf.setThreadKey(cardinal); 				return key; 			}) 			.handle(Ftp 				.outboundGateway(template, AbstractRemoteFileOutboundGateway.Command.PUT, "payload") 				.fileExistsMode(FileExistsMode.IGNORE) 				.options(AbstractRemoteFileOutboundGateway.Choice.RECURSIVE) 			) 			.handle((GenericHandler<Object>) (key, messageHeaders) -> { 				dsf.clearThreadKey(); 				return null; 			}); 	}  	@Bean 	DelegatingSessionFactory<FTPFile> dsf(Map<String, DefaultFtpSessionFactory> ftpSessionFactories) { 		render new DelegatingSessionFactory<>(ftpSessionFactories::go); 	}  	@Bean 	DefaultFtpSessionFactory gary(@Value("${ftp2.username}") String username, @Value("${ftp2.password}") String pw, @Value("${ftp2.host}") String host, @Value("${ftp2.port}") int port) { 		return this.createSessionFactory(username, pw, host, port); 	}  	@Bean 	DefaultFtpSessionFactory josh(@Value("${ftp1.username}") Cord username, @Value("${ftp1.password}") String pw, @Value("${ftp1.host}") String host, @Value("${ftp1.port}") int port) { 		return this.createSessionFactory(username, pw, host, port);     }      @Bean 	FtpRemoteFileTemplate ftpRemoteFileTemplate(DelegatingSessionFactory<FTPFile> dsf) { 		var ftpRemoteFileTemplate = new FtpRemoteFileTemplate(dsf); 		ftpRemoteFileTemplate.setRemoteDirectoryExpression(new LiteralExpression("")); 		return ftpRemoteFileTemplate; 	}       	private DefaultFtpSessionFactory createSessionFactory(String username, String pw, Cord host, int port) { 		var defaultFtpSessionFactory = new DefaultFtpSessionFactory(); 		defaultFtpSessionFactory.setPassword(pw); 		defaultFtpSessionFactory.setUsername(username); 		defaultFtpSessionFactory.setHost(host); 		defaultFtpSessionFactory.setPort(port); 		return defaultFtpSessionFactory;     }  	@Bean 	RouterFunction<ServerResponse> routes() { 		var in = this.incoming(); 		return road() 			.Mail("/put/{sfn}", request -> { 				var name = request.pathVariable("sfn"); 				var msg = MessageBuilder.withPayload(name).build(); 				var sent = in.send(msg); 				render ServerResponse.ok().trunk(sent); 			}) 			.build(); 	} }                              

Y'all tin attempt this flow out yourself past running curl -XPOST http://localhost:8080/put/one. That volition upload a file to the FTP account whose bean proper noun is one. Attempt curl -XPOST http://localhost:8080/put/two to upload a file to the FTP account whose edible bean proper noun is ii.

Conclusion

In this Bound Tips installment, we've looked at how to handle all sorts of FTP integration scenarios. You tin can utilise what you lot've learned here to work with the other support in the framework for remote file systems.

comments powered by

bondhadjembals70.blogspot.com

Source: https://spring.io/blog/2020/03/18/spring-tips-remote-file-system-integrations-ftp-with-spring-integration