How to Extract Links from a Website using Java

on February 17, 2017

Post Views: 2,465

Extract Links from a Website using Java

In this example, We will show you simple program about, How to extract links from a website using Java. The example has been tested and output shared in the same post.

Project Structure

How to Extract Links from a website using Java

Example Program

package com.dineshkrish;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.select.Elements;
/**
* 
* @author Dinesh Krishnan
*
*/
public class Example {
public static void main(String[] args) {
try {
URL url = new URL("https://idineshkrishnan.com");
Document document = Jsoup.parse(url, 5000);
int size = document.getElementsByTag("a").size();
for (int i = 0; i < size; i++) {
Elements elements = document.getElementsByTag("a");
System.out.println(elements.get(i).attr("href"));
}
} catch (MalformedURLException e) {
e.printStackTrace();
System.out.println(e.getMessage());
} catch (IOException e) {
e.printStackTrace();
System.out.println(e.getMessage());
}
}
}

Maven Dependency (pom.xml)

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.dineshkrish</groupId>
<artifactId>CommonCollection</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.10.2</version>
</dependency>
</dependencies>
</project>

Output

ttp://idineshkrishnan.com/log4j-tutorial/hello-world-example-using-log4j/
https://idineshkrishnan.com/javascript/jquery/add-or-remove-class-using-jquery/
https://idineshkrishnan.com/javascript/jquery/change-the-color-of-text-in-html-using-jquery/
https://idineshkrishnan.com/java-rmi-tutorial/call-rmi-service-using-servlet-application/
https://idineshkrishnan.com/javascript/change-opacity-of-html-element-using-javascript/
https://idineshkrishnan.com/web/css3/center-a-div-box-in-the-page-using-css/
https://idineshkrishnan.com/javascript/node-js/create-http-server-in-node-js/

find class belongs to which JAR in Eclipse

https://idineshkrishnan.com/interview-program/java-interview-program-questions/reverse-integer-array-in-java/
https://idineshkrishnan.com/java/join-two-array-using-single-loop-java/
https://idineshkrishnan.com/java/java-collections/convert-enumeration-to-arraylist-in-java/
https://idineshkrishnan.com/java/java-collections/find-minimum-maximum-value-integer-list-java/
https://idineshkrishnan.com/java/java-collections/convert-arraylist-to-enumeration-in-java/
https://idineshkrishnan.com/java/java-collections/read-only-collection-in-java/
https://idineshkrishnan.com/java/java-collections/read-only-map-in-java/
https://idineshkrishnan.com/java/java-collections/read-only-set-in-java/
https://idineshkrishnan.com/java/j2ee/simple-registration-application-using-servlet/
https://idineshkrishnan.com/java/freetts-tutorial/convert-text-to-speech-using-freetts-api/
https://idineshkrishnan.com/java-web-services/soap-tutorial/publish-and-test-simple-soap-web-service-in-java/
https://idineshkrishnan.com/java/java-io-tutorials/how-to-create-directory-in-java/
https://idineshkrishnan.com/java-rmi-tutorial/simple-login-application-in-java-rmi/
https://idineshkrishnan.com/java/j2ee/simple-login-application-using-servlet/
https://idineshkrishnan.com/java/java-collections/convert-primitive-array-to-list-object-in-java/
https://idineshkrishnan.com/java/freetts-tutorial/convert-file-text-to-speech-using-freetts-api/
https://idineshkrishnan.com/frameworks/hibernate/embedded-and-embeddable-example/
https://idineshkrishnan.com/java/utilities/get-image-width-and-height-using-java/
https://idineshkrishnan.com/java/core-java/generating-password-using-java/
https://idineshkrishnan.com/frameworks/spring-tutorial/requestparam-example-in-spring-mvc-framework/
https://idineshkrishnan.com/java/swing/center-a-jframe-window-in-java/
https://idineshkrishnan.com/java/utilities/check-words-spelling-in-java-using-jazzy-api/
https://idineshkrishnan.com/java/utilities/generate-qr-code-using-java/
https://idineshkrishnan.com/itextpdf-tutorial/how-to-convert-text-to-pdf-document-in-java/
https://idineshkrishnan.com/javascript/jquery/dynamically-add-remove-rows-html-table/
https://idineshkrishnan.com/java/java-collections/shuffle-list-in-java/
https://idineshkrishnan.com/java/gson-tutorial/covert-list-to-json-in-java/
https://idineshkrishnan.com/java/core-java/try-with-resource-statement-in-java/
https://idineshkrishnan.com
More….