Indexing of manual pages in search engines[Solved]

[theinvisible]

It seems OsclassWizards theme puts

Code: [Select]

<meta name="robots" content="noindex, nofollow, noarchive" />
<meta name="googlebot" content="noindex, nofollow, noarchive" />
in those pages.

Bender also.

That i don't know. I'm not coding geek. I want those static and some other pages like user registration and login, also popular search patterns to be indexed. I'm attaching screenshots.
When u guys found a way, lemme know.

Note: Osclasswizards is a fork of bender. They told me. They altered the original code of bender and created osclasswizards free theme.

[Aficionado]

For the moment, go to Theme's

page.php

and rem-out like this

Code: [Select]

// osc_add_hook('header','osclasswizards_nofollow_construct');

[Aficionado]

Once again this shows how poor is the very-basic SEO rules of Osclass.

Why in the WORLD apply noindex to all manual pages like About, Contact etc etc.

Still remember a post from OSclass team 3 years ago, about some "team" of SEO experts that will review and fix Osclass. Never happened. We are now in 2017 almost. Go figure.

[SmaRTeY]

Hi,

I took a look at the code and this is what Bender shows in functions.php:

Code: [Select]

        /**
        * Hook for header, meta tags robots nofollos
        */
        function bender_nofollow_construct() {
            echo '<meta name="robots" content="noindex, nofollow, noarchive" />' . PHP_EOL;
            echo '<meta name="googlebot" content="noindex, nofollow, noarchive" />' . PHP_EOL;

        }

I am not 100% sure but I THINK that the second line is what is causing googlebot from skipping the website?
The first line tells robots what subfolders not to index and should be left alone (or improved is this is causing the skip of manual pages), is correct and removing that will let Google also index all other files which is something you do not want for security reasons.


Regards,
Eric

[Aficionado]

What puzzles me is this kind of page, not a manual page. Why it is also tagged with NO-everything ?

[Aficionado]

Hi,

I took a look at the code and this is what Bender shows in functions.php:

Code: [Select]

        /**
        * Hook for header, meta tags robots nofollos
        */
        function bender_nofollow_construct() {
            echo '<meta name="robots" content="noindex, nofollow, noarchive" />' . PHP_EOL;
            echo '<meta name="googlebot" content="noindex, nofollow, noarchive" />' . PHP_EOL;

        }

I am not 100% sure but I THINK that the second line is what is causing googlebot from skipping the website?
The first line tells robots what subfolders not to index and should be left alone (or improved is this is causing the skip of manual pages), is correct and removing that will let Google also index all other files which is something you do not want for security reasons.


Regards,
Eric

I think better just remove this from page.php of Bender (and other Bender-based themes).

Code: [Select]

    // meta tag robots
    osc_add_hook('header','bender_nofollow_construct');

[SmaRTeY]

Well, there is a thought of preventing certain pages form being indexed/crawled to keep those private. Also there can be reason not to index ad pages in case the ads get deleted before the index is updated which would cause ending up with a 404 not found page.
In the end it is a personal choice I think together with some security in mind. I do think SEO can be improved in Osclass yet there also is a commercial plugin available in Market. In the end besides the Bender theme all other theme creators have their own responsibility in making their themes as SEO friendly as possible not only Osclass for this matter.

What puzzles me is this kind of page, not a manual page. Why it is also tagged with NO-everything ?

[SmaRTeY]

That will certainly get all the bots visiting, indexing and showing your website pages

EDIT: Before I forget, the robots'txt file in website root folder also directs robots how to go about indexing/crawling.
https://support.google.com/webmasters/answer/6062608?hl=en

With this in mind it seems that removing the lines you mention will save page building time and only robots.txt would be used by bots still keeping certain files 'private'. With this in mind I also wonder why the metatags are used as well....

Hi,

I took a look at the code and this is what Bender shows in functions.php:

Code: [Select]

        /**
        * Hook for header, meta tags robots nofollos
        */
        function bender_nofollow_construct() {
            echo '<meta name="robots" content="noindex, nofollow, noarchive" />' . PHP_EOL;
            echo '<meta name="googlebot" content="noindex, nofollow, noarchive" />' . PHP_EOL;

        }

I am not 100% sure but I THINK that the second line is what is causing googlebot from skipping the website?
The first line tells robots what subfolders not to index and should be left alone (or improved is this is causing the skip of manual pages), is correct and removing that will let Google also index all other files which is something you do not want for security reasons.


Regards,
Eric

I think better just remove this from page.php of Bender (and other Bender-based themes).

Code: [Select]

    // meta tag robots
    osc_add_hook('header','bender_nofollow_construct');

[Aficionado]

That will certainly get all the bots visiting, indexing and showing your website pages

I think better just remove this from page.php of Bender (and other Bender-based themes).

Code: [Select]

    // meta tag robots
    osc_add_hook('header','bender_nofollow_construct');

No i think this is only for static manual pages.

[Aficionado]

Well, there is a thought of preventing certain pages form being indexed/crawled to keep those private. Also there can be reason not to index ad pages in case the ads get deleted before the index is updated which would cause ending up with a 404 not found page.
In the end it is a personal choice I think together with some security in mind. I do think SEO can be improved in Osclass yet there also is a commercial plugin available in Market. In the end besides the Bender theme all other theme creators have their own responsibility in making their themes as SEO friendly as possible not only Osclass for this matter.

I disagree. I'm talking about the very-very basic seo that all scripts should follow. Proper titles tags everywhere for example. Proper Descriptions everywhere since GOOGLE use descriptions (no matter some crappy seo experts say). Things like that.

Take a look at Wordpress. It is almost basic-seo ready out of the box. Sure, if you want more, there are several plugins. But not needed imho.

Still, i don't understand why HIDE some manual static pages like About, Contact etc. I don't get it at all.

[SmaRTeY]

A quick scan using Boxer theme shows:
contact.php / page.php and search.php as well as all user pages

EDIT: meaning only remove the hook from contact.php and page.php unless you also want all user related pages to be indexed/crawled

That will certainly get all the bots visiting, indexing and showing your website pages

I think better just remove this from page.php of Bender (and other Bender-based themes).

Code: [Select]

    // meta tag robots
    osc_add_hook('header','bender_nofollow_construct');

No i think this is only for static manual pages.

[SmaRTeY]

We agree Aficionado; "I do think SEO can be improved in Osclass" and I also have some questionmarks here.
BUT, removing the hook will make search engines only use the robots.txt, that is if you have it.

Well, there is a thought of preventing certain pages form being indexed/crawled to keep those private. Also there can be reason not to index ad pages in case the ads get deleted before the index is updated which would cause ending up with a 404 not found page.
In the end it is a personal choice I think together with some security in mind. I do think SEO can be improved in Osclass yet there also is a commercial plugin available in Market. In the end besides the Bender theme all other theme creators have their own responsibility in making their themes as SEO friendly as possible not only Osclass for this matter.

I disagree. I'm talking about the very-very basic seo that all scripts should follow. Proper titles tags everywhere for example. Proper Descriptions everywhere since GOOGLE use descriptions (no matter some crappy seo experts say). Things like that.

Take a look at Wordpress. It is almost basic-seo ready out of the box. Sure, if you want more, there are several plugins. But not needed imho.

Still, i don't understand why HIDE some manual static pages like About, Contact etc. I don't get it at all.

[Aficionado]

Eric nice catch about the contact.php.

As for users, are you sure, since users profiles seem indexed .... Or you mean something else ?

[SmaRTeY]

Well, I noticed that many user-xxx.php files also contain the add_hook... but I didn't check the public user profile specificly. If you see the profiles indexed that php file will not contain the hook, the other user related files do make sense not to be indexed/crawled.

Eric nice catch about the contact.php.

As for users, are you sure, since users profiles seem indexed .... Or you mean something else ?

[theinvisible]

A quick scan using Boxer theme shows:
contact.php / page.php and search.php as well as all user pages

EDIT: meaning only remove the hook from contact.php and page.php unless you also want all user related pages to be indexed/crawled

That will certainly get all the bots visiting, indexing and showing your website pages

I think better just remove this from page.php of Bender (and other Bender-based themes).

Code: [Select]

    // meta tag robots
    osc_add_hook('header','bender_nofollow_construct');

No i think this is only for static manual pages.

User profiles are indexed i think, Eric. check this screenshot.