Last time I dug the site, I shared the use of Huginn to capture the RSS and WeChat public account updates of any website. Many friends left messages saying that Huginn configuration is too difficult, and there is no ready-made scenario library. Many crawling rules need to be explored and customized by yourself. Definition is relatively difficult. The most important thing is that you need to pay a hosting fee yourself to build Huginn.
The Feed43 I’m sharing today is equivalent to Huginn’s RSS crawling and subscription function. Huginn has powerful functions. If you just want to use Huginn's RSS subscription, you might as well switch to Feed43, a free online service for subscribing to updates from any website. Regardless of whether you are using a website, just web pages that can be accessed with a browser can be subscribed to Feed43.
Feed43 has a free plan and is crawled every 6 hours, so Feed43 is not suitable for subscribing to websites that update frequently. Feed43 can generate RSS feeds from any website, and then you can put this feed in your own RSS reader, or you can also cooperate with iFTTT, fivefilters, and freefullrss to achieve RSS full-text reading, email notifications, and mobile APP reminders.
For more practical tools, you can also check out:
- Three command tools Rsync, SCP, and Tar-quick solution to VPS remote website relocation and data synchronization
- Use Huginn to capture RSS and WeChat public account updates from any website - create a one-stop information reading platform
- Three free tools to help you detect the authenticity of VPS servers - VPS host performance and speed test methods
PS: Updated on January 31, 2018. In addition to using third-party RSS subscription services, friends who have their own servers or hosts can also build their own RSS readers. Reference: Two excellent open source RSS Reader tools: Miniflux and Tiny Tiny RSS-self-built online RSS reader.
1. Use Feed43 to subscribe to RSS of any website.
Official website:
- HTTP://feed43.com/
To enter the website, register an account first. Feed43 supports managing and editing your own feeds at any time.
Then, you can add the websites you want to subscribe to. If there are garbled characters, fill in the following Encoding: utf-8 or gbk.
Then, Feed43 will output the Html source code of the web page. Fill in the rules you want to crawl at Item (repeatable) Search Pattern*:. There are two parameters: {%} and {*}. {%} represents a variable, {*} represents omission. Observe carefully, if we want to grab the title, copy the Html tag of the title, replace the variables, and omit the ones that are not needed.
Refer to the rules of the latest articles on the homepage of wzfou.com that I grabbed, as shown below:
If there is no content after clicking Extract, you can start with the simplest tags, such as grabbing only the H tags first, and then add other content tags. After confirming, it is time to determine the RSS template, mainly the title, content, and links. Fill in the contents based on the {%} captured above.
Finally, click Preview and you can see that the RSS feed has been successfully created.
Open this .xml address with a browser, and we can see the updated content of RSS. You can subscribe to this RSS address with an RSS reader.
Of course, you can also modify or delete this RSS feed at any time.
2. Free and self-made RSS Feed full text reading
2.1 RSS full text subscription online website
As long as you fill in the RSS subscription address, you can use the following websites to subscribe to RSS full text:
- http://fetch RSS.com
- http://five filters.org
- HTTP://full content RSS.com/
- HTTPS://wuwuwu.free full RSS.com/
Among them, freefullrss is completely free, enter your RSS subscription address.
Then, freefullrss will generate the RSS full text.
You can use this address to subscribe to an RSS reader. What you see in the RSS reader is the full text of our subscription. (Click to enlarge)
2.2 Self-made RSS full-text subscription service
project address:
- bitbucket: https://bitbucket.org/fivefilters/full-text-rss
- Github: https://github.com/Dither/full-text-rss
The bitbucket project of Full-Text RSS seems to have been updated. You can directly use git clone https://bitbucket.org/fivefilters/full-text-rss.git to download the latest one. Full-Text RSS is just a PHP program and does not require a database. So after you download Full-Text RSS, decompress it and upload it directly to the server for sending.
Then, open the URL and you can enter the Full-Text RSS interface and enter the RSS address where you want to obtain the full text.
After parsing, you will see the full text of the RSS in the browser.
Full-Text RSS also provides the function of crawling the RSS of any website. However, you need to formulate the crawling rules first. The Full-Text RSS official website also provides thousands of website rules made by netizens themselves. Some are in Chinese and we can use them directly. the way is:
- Enter the directory: site_config/standard/
- Delete all contents in the directory
- Execute command:
git clone https://github.com/fivefilters/ftr-site-config.git . - Git will start downloading the latest site config file
- If you want to update, just repeat the above operation, or you can use
git pull.
Of course, we can also use the online generated RSS feed provided by Full-Text RSS: http://createfeed.fivefilters.org/, enter the website address you want to subscribe to.
Then select the web page to update the content. At this time, fivefilters will generate website crawling rules. You can just click to download.
Upload the downloaded configuration file to site_config/standard/.
3. IFTTT: Implement RSS email notifications and APP reminders
The combination of RSS subscription and IFTTT will become very powerful. IFTTT can push the content of RSS subscription to your email, mobile phone, computer and other terminal devices.
After adding the RSS subscription address in IFTTT, you can then receive reminders in the mobile APP.
In addition, Blogtrottr supports keyword filtering and pushing RSS, so that you will only receive RSS subscription content that interests you: https://blogtrottr.com/.
4. Summary
Feed43 is a very powerful crawling tool that can subscribe to any website and convert it into an RSS address. Unfortunately, free users can only update it every 6 hours. Free RSS Feed full-text reading services have many limitations, and some services are not very stable yet, so it is better to build one yourself.
So far, the only problem is that we hope Feed43 does not shut down. After all, it is a free third-party service, and RSS is not in recession now. The one-and-done way to dig up a website is to use Huginn to grab RSS and WeChat official account updates from any website - creating a one-stop information reading platform.