Register - Login
Views: 96132530
Main - Memberlist - Active users - Calendar - Wiki - IRC Chat - Online users
Ranks - Rules/FAQ - Stats - Latest Posts - Color Chart - Smilies
12-10-18 06:57:31 PM

Jul - Meta - Search indexing New poll - New thread - New reply
Pages: 1 2Next newer thread | Next older thread
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-10-18 06:16:08 PM Link | Quote
I disabled search indexing for these forums.

They have been "noindex" for a while, but I've now set it to disallow everything, via header, meta tags, and robots.txt.

If you have any questions, feel free to ask.
Tarale
Catgirl
C:\ DOS
C:\ DOS RUN
RUN DOS RUN
Level: 83


Posts: 1836/1881
EXP: 5260979
For next: 171242

Since: 07-23-07

Pronouns: she/her

Since last post: 9 days
Last activity: 5 days

Posted on 09-10-18 06:21:20 PM Link | Quote
As in, the goooooogle?

I am more than cool with
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-10-18 06:24:26 PM Link | Quote
The google, the bing, the yahoo. Every robot that respects one of those.
Kak
heh
Level: 73


Posts: 1806/1817
EXP: 3397762
For next: 88106

Since: 09-03-13

From: ???

Since last post: 13 days
Last activity: 1 hour

Posted on 09-10-18 06:30:37 PM Link | Quote
<+Schezo[L]> does this mean search.php will become a reality?
* +Schezo[L] runs

kind of a shame as there's no search functionality in the code, but I guess this is a way to solve the issue of some forums getting cached
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-10-18 07:06:20 PM Link | Quote
The search tool would be nice to have, admittedly. Maybe something can be figured out.
Nksor

Beezo
Level: 43


Posts: 476/478
EXP: 551496
For next: 13550

Since: 04-26-11

Pronouns: they/them
From: olympia, wa

Since last post: 90 days
Last activity: 77 days

Posted on 09-10-18 08:51:38 PM Link | Quote
Out of curiosity, why?
NinLenna

Cheep-cheep
Level: 25


Posts: 157/183
EXP: 80864
For next: 8756

Since: 01-08-16

Pronouns: she/her
From: Dimension X Pipe

Since last post: 6 days
Last activity: 1 hour

Posted on 09-10-18 08:55:19 PM (last edited by Nin★Collin at 09-10-18 08:56:14 PM) Link | Quote
I have search indexing disabled on my website. However, I do allow Wayback Machine to crawl it.

I do like the sense of privacy of having it disabled here.
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-10-18 08:57:37 PM Link | Quote
Originally posted by Nksor
Out of curiosity, why?

In general, I don't feel like having this place searchable — it contains a lot of old, old things, and with how the internet is these days it just isn't good to have this sort of history up any more.
RanAS
Member
Level: 41


Posts: 459/521
EXP: 463990
For next: 16155

Since: 10-10-14

From: Brazil

Since last post: 16 hours
Last activity: 4 hours

Posted on 09-10-18 09:04:48 PM Link | Quote
I agree with this, noindex/nofollow seems better for Jul.

It appears Google's search results have already updated and aren't showing any results from Jul! I assume Bing and DuckDuckGo will probably update their results later this week.
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-10-18 10:57:43 PM Link | Quote
I used the "request unlisting" feature on Goog's site control thing, so that will be why everything vanished.
Nksor

Beezo
Level: 43


Posts: 478/478
EXP: 551496
For next: 13550

Since: 04-26-11

Pronouns: they/them
From: olympia, wa

Since last post: 90 days
Last activity: 77 days

Posted on 09-11-18 12:04:07 AM Link | Quote
Originally posted by Xkeeper
Originally posted by Nksor
Out of curiosity, why?

In general, I don't feel like having this place searchable — it contains a lot of old, old things, and with how the internet is these days it just isn't good to have this sort of history up any more.

Makes sense! I second the request for some way to search the site for the sake of finding posts/threads/users/etc, since I've used "site:jul.rustedlogic.net" to find things on here in the past.
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-11-18 12:58:33 AM Link | Quote
I'll try to figure something out (the code is open source, so someone can always contribute one ). In the worst case, you can ask me to try to find something for you.
Q
Mis7eryMyra
Level: 100


Posts: 2884/2913
EXP: 10125667
For next: 228729

Since: 08-02-07

Pronouns: she/her
From: Nowhere

Since last post: 2 days
Last activity: 8 hours

Posted on 09-11-18 12:58:47 AM Link | Quote
rip Julooh

It's probably for the best, though
NinLenna

Cheep-cheep
Level: 25


Posts: 158/183
EXP: 80864
For next: 8756

Since: 01-08-16

Pronouns: she/her
From: Dimension X Pipe

Since last post: 6 days
Last activity: 1 hour

Posted on 09-11-18 08:33:20 AM Link | Quote
Someone could try and port over search.php from 1.92.
I've never looked at the code to see how bad it is though.
Lunaria

Moon Bunny! :3
Level: 133


Posts: 5594/5623
EXP: 27173813
For next: 678854

Since: 07-28-07

Pronouns: she/her
From: pile of fluff

Since last post: 8 hours
Last activity: 8 hours

Posted on 09-11-18 09:12:17 AM Link | Quote
Good change, tbh. Not that I have much faith in those places respecting at all being told to sod off, but yeah.

I feel that the people who would find this place by searching for it in google/etc or someone's name... are more likely than not the kind of people that probably shouldn't be here. I like having places on the net that aren't toxic and preferably places those kind of individuals can't find, or at least will have a harder time with. idk.
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-11-18 09:24:40 AM Link | Quote
More or less. For the more annoying bots that haven't taken the hint, I've been blocking them at a server level based on their user agent. Not really foolproof by any means, but it's something.
Halian

six legged drgn whomst build worlds
Level: 44


Posts: 467/480
EXP: 585075
For next: 26210

Since: 06-20-10

Pronouns: he/him
From: Central Florida

Since last post: 10 days
Last activity: 1 day

Posted on 09-11-18 09:37:18 AM Link | Quote
But the Wayback Machine
Kak
heh
Level: 73


Posts: 1807/1817
EXP: 3397762
For next: 88106

Since: 09-03-13

From: ???

Since last post: 13 days
Last activity: 1 hour

Posted on 09-11-18 12:59:18 PM (last edited by Kak at 09-11-18 01:03:05 PM) Link | Quote
Originally posted by Halian
But the Wayback Machine
unless they changed it again, the wayback machine was updated to (finally) ignore robots.txt some time ago. too many dead websites with history lost thanks to domain names being bought with a default robots.txt blocking everything, among other fun issues

which means that this shouldn't be a problem
Xkeeper

Level: 251


Posts: 23358/24710
EXP: 251088372
For next: 2951504

Since: 07-03-07

Pronouns: they/them, she/her, etc.

Since last post: 19 hours
Last activity: 5 hours

Posted on 09-11-18 01:42:40 PM Link | Quote
I can add an exception for the Wayback Machine because I agree it's nice to have, and unlike everything else you can't just do a random text search on it.
NinLenna

Cheep-cheep
Level: 25


Posts: 159/183
EXP: 80864
For next: 8756

Since: 01-08-16

Pronouns: she/her
From: Dimension X Pipe

Since last post: 6 days
Last activity: 1 hour

Posted on 09-11-18 02:06:21 PM Link | Quote
Originally posted by Kak
Originally posted by Halian
But the Wayback Machine
unless they changed it again, the wayback machine was updated to (finally) ignore robots.txt some time ago. too many dead websites with history lost thanks to domain names being bought with a default robots.txt blocking everything, among other fun issues

which means that this shouldn't be a problem

Then they did change it again. I had this issue with my website a couple days ago when I added a robots.txt file to another of my websites.
Pages: 1 2Next newer thread | Next older thread
Jul - Meta - Search indexing New poll - New thread - New reply




Rusted Logic

Acmlmboard - commit 220d144 [2018-11-04]
©2000-2018 Acmlm, Xkeeper, Inuyasha, et al.

30 database queries, 3 query cache hits.
Query execution time: 0.235614 seconds
Script execution time: 0.027153 seconds
Total render time: 0.262767 seconds