Search Problem OC 3.5.3

[SmaRTeY]

  Hi teseo.....today some testing time since I NEED this to work independant of used locale language.
I am thinking since this is by design there's no bug to report but more a work around making it independant.....

Wish me luck!

[SmaRTeY]

Dude!
You forgot the  $mSearch->addGroupBy("pk_i_id"); again in the second third code sample (relevance) so when I copied the 3rd code it was showing me funny results again. 

Haha.... after I added that code (again) the duplicate was gone.

Okay here's the deal, not working 100% yet.
My test setup = 2 active languages on front-end, all ads but one posted in 1 language, the other one is posted in 2nd language.

Now the search works for each chosen language which is perfect! Meaning it no longer matters in what language an ad is posted and whatever language is chosen by a user.

BUT when saving a search alert it is showing me the alert WITH the ads in the language the ads were posted.
Once I select the other language the alert is showing WITHOUT the ads 

Probably meaning.... there goes yet another saturday? 

Nonetheless, I am pretty happy with the search working now independant of the chosen language either searching or posting ads.

Regards,
Eric

[SmaRTeY]

Teseo, this thread is about the same 'issue' and works flawlessly:
[/url]http://forums.osclass.org/general-help/search-doesn't-work-on-multi-language-site/?topicseen[/url]

Since there seems to be a nr. of corrections to be made which do not feel very stable I am thinking about using an class 'extend' and create a new search class which is missing the locale condition as mentioned in the thread. Using the new class also prevents future update issues, what do you think? 

Regards,
Eric

[teseo]

Hi,

Sorry, I won't take a look at that while a solution not involving core files modification is possible. And I think I've managed to get it, look now my 4th version:

http://forums.osclass.org/3-5-x/search-problem-oc-3-5-3/msg113778/#msg113778

Problem was, again, that locale isn't stored on alerts table, so I had to "fool" the User Alerts page about the user's current language. Function cust_alerts_locale_hack_start retrieves the current one, stores it in a View variable and set it to '%'. Just after the alerts has been retrieved from database, cust_alerts_locale_hack_end restores correct language so the page won't always use English.

While at it, I found a way to avoid duplicates and retrieve 3 different ads every time, replacing the blessed "GROUP BY" by a DISTINCT inside the subquery.

Next saturday I will be on the beach, don't count me in... 

Regards

[SmaRTeY]

Hi teseo,

so I've added now a GROUP BY to prevent that

you must have a 'group by' blindspot 

Your fourth solution...... sorry but it gave me doubles in a text search..... BUT when actually adding the group by

Code: [Select]

$mSearch->addGroupBy("pk_i_id"); in function cust_refined_pattern_search it looks okay. I thought the DISTINCT in sql was taking care of the doubles but it didn't and I haven't given it more thought.

So far I checked the Alert saving/display and it works afaik and I also tested a search (text) with again no duplicates.
IF I see a miss the coming weeks I'll let you know but besides the group by this one really looks good! 

P.s. my earlier referral to a core fix looked to be okay but also had doubles issue in the text search 


Thanks,
Eric

***UPDATED with 4th version***

Well, first, I've realized that my previous solution to include all languages in the search could retrieve same ad several times, so I've added now a GROUP BY to prevent that happening.

And regarding your request for including all languages plus relevance ordering, here it's the solution (of course, use one or another):

4th version (Alerts compatible -no duplicate ads, Multilanguage on User Alerts)

Code: [Select]

<?php
function cust_refined_pattern_search($params) {

    if ($params['sPattern'] != '') {
        $mSearch = Search::newInstance();
        $mSearch->addJoinTable( null, sprintf("(SELECT DISTINCT(fk_i_item_id), MATCH(s_title, s_description) AGAINST('%s') AS relevance FROM oc_t_item_description ) des", $params['sPattern']), 'des.fk_i_item_id = oc_t_item.pk_i_id', 'INNER' );
        $mSearch->addLocale('%');
        $mSearch->order("des.relevance DESC, dt_pub_date", "DESC");
    }
}

osc_add_hook('search_conditions', 'cust_refined_pattern_search');

function cust_alerts_locale_hack_start() {
    
    if (osc_get_osclass_section() == "alerts") {
        View::newInstance()->_exportVariableToView('aUserLocale', Session::newInstance()->_get('userLocale'));
        Session::newInstance()->_set('userLocale', '%');
    }
}

osc_add_hook('init', 'cust_alerts_locale_hack_start'); 

function cust_alerts_locale_hack_end($conditions) {

    if (osc_get_osclass_section() == "alerts") {
        Session::newInstance()->_set('userLocale', View::newInstance()->_get('aUserLocale'));
    }
    
    return $conditions;
}

osc_add_filter('sql_search_item_conditions', 'cust_alerts_locale_hack_end');
?>

3rd version (Alerts compatible -no duplicate ads, No multilanguage on User Alerts)

Code: [Select]

<?php
function cust_refined_pattern_search($params) {

    $mSearch =  Search::newInstance();

    if ($params['sPattern'] != '') {
        $mSearch->addJoinTable( null, sprintf("(SELECT fk_i_item_id, MATCH(s_title, s_description) AGAINST('%s') AS relevance FROM oc_t_item_description ) des", $params['sPattern']), 'des.fk_i_item_id = oc_t_item.pk_i_id', 'INNER' );
        $mSearch->addLocale('%');
        $mSearch->addGroupBy("pk_i_id");
        $mSearch->order("des.relevance DESC, dt_pub_date", "DESC");
    }
}

osc_add_hook('search_conditions', 'cust_refined_pattern_search');

function cust_clean_duplicate_ads_user_alerts() {
    
    $clean_alerts = View::newInstance()->_get('alerts');

    if ($clean_alerts) {
        foreach ($clean_alerts as $key => $alert) {

            $outputArray = array();
            $keysArray = array();
            foreach ($clean_alerts[$key]['items'] as $innerArray) {
                if (!in_array($innerArray['pk_i_id'], $keysArray)) {
                    $keysArray[] = $innerArray['pk_i_id'];
                    $outputArray[] = $innerArray;
                }
            }

            $clean_alerts[$key]['items'] = $outputArray;
        }

    View::newInstance()->_exportVariableToView('alerts', $clean_alerts);
    }  
}

osc_add_hook('before_html', 'cust_clean_duplicate_ads_user_alerts');
?>

2nd version (Alerts compatible -but including duplicate ads under certain circumstances)

Code: [Select]

<?php
function cust_refined_pattern_search($params) {

    $mSearch =  Search::newInstance();

    if ($params['sPattern'] != '') {
        $mSearch->addJoinTable( null, sprintf("(SELECT fk_i_item_id, MATCH(s_title, s_description) AGAINST('%s') AS relevance FROM oc_t_item_description ) des", $params['sPattern']), 'des.fk_i_item_id = oc_t_item.pk_i_id', 'INNER' );
        $mSearch->addLocale('%');
        $mSearch->order("des.relevance DESC, dt_pub_date", "DESC");
    }
}

osc_add_hook('search_conditions', 'cust_refined_pattern_search');
?>
1st version (Alerts uncompatible)

Code: [Select]

<?php
function cust_refined_pattern_search($params) {
    $mSearch =  Search::newInstance();
    
    if ($params['sPattern'] != '') {
        $mSearch->addField( sprintf("MATCH(d.s_title, d.s_description) AGAINST('%s') AS relevance", $params['sPattern']) );
        $mSearch->addLocale('%');
        $mSearch->order("relevance DESC, dt_pub_date", "DESC");
        $mSearch->addGroupBy("pk_i_id");
    }
}

osc_add_hook('search_conditions', 'cust_refined_pattern_search');
?>
Regards

[SmaRTeY]

Don't want to be a pain but I think I found another 'issue' which is related to this new function:

Code: [Select]

function cust_alerts_locale_hack_start() {
    if (osc_get_osclass_section() == "alerts") {
        View::newInstance()->_exportVariableToView('aUserLocale', Session::newInstance()->_get('userLocale'));
        Session::newInstance()->_set('userLocale', '%');
    }
}
osc_add_hook('init', 'cust_alerts_locale_hack_start');
It seems that the _set('userLocale','%'); code is affecting the function 'osc_get_current_user_locale' (probably other locale functions as well) which is used to determine the user's chosen locale. Meaning, the new function is actually disrupting getting to know what language was chosen by the user which in my case causes a failure in my header where I show the chosen language. It can be a while but then all of a sudden the user locale is set to % and I can see it being returned by the function 'osc_get_current_user_locale'.

So the locale is having multi purpose use....1 when querying the items, 2 with Alerts and 3 keeping track of default/chosen language in Osclass. It does make sense but it also means you need to find a way to keep chosen language stick to 1 language and have the query locale changed to % (as well as the alerts).

EDIT:
I didn't look at the other function yet, the 'cust_alerts_locale_hack_end' which I think you use to set the locale to the chosen language again. Having said so I tend to think now that either the used hook is incorrect or is not working a 100% seeing my locale get 'lost' (which I have not been able to pin point yet).


Regards,
Eric

[teseo]

I'm afraid that's not the worst news...

so I've added now a GROUP BY to prevent that

you must have a 'group by' blindspot 

If you [still ] can recall, the goal was to eliminate that GROUP BY because it isn't stored for alerts. I thought I had achieved that replacing it by that SELECT DISTINCT(fk_i_item_id) in the JOIN subquery, but obviously not. See, this would be the perfect query:

SELECT DISTINCT(oc_t_item.pk_i_id) as nodups, oc_t_item.*, oc_t_item.s_contact_name as s_user_name
FROM (oc_t_item)
LEFT JOIN oc_t_item_description as d ON d.fk_i_item_id = oc_t_item.pk_i_id
INNER JOIN (SELECT DISTINCT(fk_i_item_id), MATCH(s_title, s_description) AGAINST('test') AS relevance FROM oc_t_item_description ) des ON des.fk_i_item_id = oc_t_item.pk_i_id
WHERE MATCH(d.s_title, d.s_description) AGAINST('test' IN BOOLEAN MODE)
AND ( d.fk_c_locale_code LIKE '%' )
AND oc_t_item.b_enabled = 1 AND oc_t_item.b_active = 1 AND oc_t_item.b_spam = 0 AND (oc_t_item.b_premium = 1 || oc_t_item.dt_expiration >= '2015-04-27 17:08:26')
ORDER BY des.relevance DESC, dt_pub_date DESC

But, no way, you may add fields for the SELECT at the end of the chain:

SELECT oc_t_item.*, oc_t_item.s_contact_name as s_user_name, DISTINCT(oc_t_item.pk_i_id) as nodups

But not at the start, and DISTINCT needs to be placed right there, just after SELECT.

So at this point we'd need to return to GROUP BY for search page query and erasing of duplicates on alert pages, just as one of my earlier versions did.

When I have a little time I'll do that and study that last issue you discovered.

Regards

[SmaRTeY]

LOL.... well don't worry about it, it's fun to learn a bit more every time and since this is pretty the 'heart' of the system after this your/our understanding will only be better.

Interesting findings! (and no hurry)


Thanks,
Eric

[SmaRTeY]

@teseo,

I had another thought regarding the multiple languages and the 'issue' we are trying to fix here well actually trying to break the design and have all ads in searches/alerts no matter what language was used when posting a new ad.

HOW ABOUT; simply setting the locale to the default preventing new ads to be saved based on the locale in use? So, no matter what locale is selected by the user, when posting a new ad we simply always use ONE locale which will be used when saving new ads. this way you do not have to make any changes to the search/alerts and we follow the design of Osclass with regard of the architecture.

Incorrect thought, with more than 1 language in use we still need the fix of getting ads in other locale but the above would make sure all ads are posted in only one language keeping the system more consistent if you want to use 1 main language yet show all in searches.

I was thinking about a simple hook before new ad is saved that changes the locale to the (default) one you want.
Do you think this is possible? 

Regards,
Eric

[SmaRTeY]

Some thoughts trying to have an overview of what happens.....

Posting & Locales
1) In good working original Osclass new ads are posted using the active locale.
2) In case someone changes language locale and starts editing his existing ad posted using another language locale when saving the edited ad is saved again using the new locale. In this case the ad is being duplicated and having differences where the owner probably doesn't know about!

Search
3) Search is checking locale mainly when doing a text search
4) In case you simply press search without criteria the system shows ALL ads independant of locales

SO, my thought is that posting and editing of ads should always stick to the locale which was used when posting the ad to prevent the creation of duplicates and creating two different ad versions in two different locales!!
IMO this could be considered a bug or a feature with nasty side effects. 

Next, to ignore the locales in text search (and maybe other criteria using locale) we need to simply take it out of the search query. This could be an Admin option and/or a core change OR this might be achieved with the search changes done by teseo using functions.php.
The latter was causing me issues BUT I also had an issue in my theme I wasn't aware of when posting new ads. One locale function being used in my header prevented it to work again in the body of the page causing my newly posted ads to be posted without the locale in the table! Causing my system to have ads with 1 locale, ad duplicates with different locale and different ad properties and ads without a locale.

As for the latter, I was able to fix my theme issue so my new ads are posted the correct way again and I will test the last solution again teseo created to see if that makes any difference as for working or not working 100%.

But most of all I want the edit of ads not create duplicates..... I am not sure if this should be considered a bug and I hope _Conejo is reading along and can shed a light on this.

All other thoughts very welcome here!

Regards,
Eric

[arcsales]

The search is very important to all websites and with osclass there is a problem like you both said.
I was trying to integrate the codes provided (version 4 and 7) but both are breaking my website languagage.
Which one I should use if I only want to bring up the most relevant ads and then all other if there is no other filter used?

Thanks in advance for your replays.

[teseo]

Hi,

http://forums.osclass.org/3-5-x/search-problem-oc-3-5-3/msg113778/#msg113778

Last version (7th).

Add that at the very bottom of your theme functions.php:
Notes:
1.- Take care not to leave blank lines after this.
2.- If your theme functions.php doesn't end with ?> skip first line of my code.

Regards

[arcsales]

Hi,

http://forums.osclass.org/3-5-x/search-problem-oc-3-5-3/msg113778/#msg113778

Last version (7th).

Add that at the very bottom of your theme functions.php:
Notes:
1.- Take care not to leave blank lines after this.
2.- If your theme functions.php doesn't end with ?> skip first line of my code.

Regards

I've done that but for some reason after I hit subscribe to this search my browser cookies are breaking and the browser does not read my translated .po but bring back US language.
If I open my website in incognito window then it looks fine.

[teseo]

Sorry, I couldn't reproduce anything of that. Maybe your theme is incompatible with this function somehow... Have you tried with another theme?

Regards

[arcsales]

No but its a good idea and I will test...the point is that you might be right partly about the theme.
When i was building my website at 1st i had a problem with the translation and some plugins.Because the plugins was not compatible to work with other than EN language. So I did cheat the system by translating the original EN files within the files.Basically the file says EN but inside is an other language so that might cause the problem as well...im not quite sure but i will do some testing with other languages.