Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S2S/listeners #4455

Open
wants to merge 10 commits into
base: feature/listeners
Choose a base branch
from
Open

S2S/listeners #4455

wants to merge 10 commits into from

Conversation

NelsonVides
Copy link
Collaborator

@NelsonVides NelsonVides commented Jan 7, 2025

This PR introduces ranch listeners to S2S, which was the last XMPP listener using our old mongoose_tcp_listener. This means we can remove this custom module together with the tests for it, and it also means that now S2S uses the same listener logic than Components and C2S, and therefore a lot of code can be extracted to be shared, most importantly in the config, but also in instrumentation.

Note the following:

  1. S2S in and out was using mongoose_transport as the transport layer, which implemented both logic to be a server as well as a client. I split this, the server logic was removed entirely from mongoose_transport, which is now the TCP client implementation for ejabberd_s2s_out.
  2. ejabberd_s2s_in got a new TCP server logic implemented in mongoose_s2s_socket. It's very similar to the socket implementation for components and c2s, but it is still running on its own gen_server instead of having been integrated into a newer gen_statem as it was done for components and c2s already, so until then, those three modules aren't refactored together yet.
  3. S2S incoming connections listener is now implemented into mongoose_s2s_listener, which is incredibly similar to mongoose_components_listener and mongoose_c2s_listener. So similar that common code between the three was extracted into mongoose_listener and instrumentation for all listeners was finally unified, no longer having weird adhoc initialisers as it was initially the case for c2s. Tests were updated to take this into account.
  4. As now all C2S, S2S, and Components, use the same listener logic and implementation details, the config was reworked, putting a lot more keys that are common in the common sections of both the spec and the documentation.

@NelsonVides NelsonVides changed the title S2s/listeners S2S/listeners Jan 7, 2025
@NelsonVides NelsonVides force-pushed the s2s/listeners branch 3 times, most recently from 97411af to b3aa6d2 Compare January 7, 2025 17:32
@mongoose-im

This comment was marked as outdated.

Copy link

codecov bot commented Jan 7, 2025

Codecov Report

Attention: Patch coverage is 85.93750% with 27 lines in your changes missing coverage. Please review.

Project coverage is 85.16%. Comparing base (d653dc2) to head (4a99ce6).
Report is 4 commits behind head on feature/listeners.

Files with missing lines Patch % Lines
src/s2s/mongoose_s2s_socket.erl 80.80% 24 Missing ⚠️
src/s2s/mongoose_s2s_info.erl 80.00% 2 Missing ⚠️
src/s2s/ejabberd_s2s_in.erl 95.00% 1 Missing ⚠️
Additional details and impacted files
@@                  Coverage Diff                  @@
##           feature/listeners    #4455      +/-   ##
=====================================================
- Coverage              85.35%   85.16%   -0.19%     
=====================================================
  Files                    553      554       +1     
  Lines                  33959    34021      +62     
=====================================================
- Hits                   28985    28975      -10     
- Misses                  4974     5046      +72     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mongoose-im
Copy link
Collaborator

mongoose-im commented Jan 7, 2025

elasticsearch_and_cassandra_27 / elasticsearch_and_cassandra_mnesia / 69764e3
Reports root/ big
OK: 472 / Failed: 0 / User-skipped: 49 / Auto-skipped: 0


small_tests_26 / small_tests / 69764e3
Reports root / small


small_tests_27 / small_tests / 69764e3
Reports root / small


small_tests_27_arm64 / small_tests / 69764e3
Reports root / small


ldap_mnesia_26 / ldap_mnesia / 69764e3
Reports root/ big
OK: 2350 / Failed: 0 / User-skipped: 912 / Auto-skipped: 0


ldap_mnesia_27 / ldap_mnesia / 69764e3
Reports root/ big
OK: 2350 / Failed: 0 / User-skipped: 912 / Auto-skipped: 0


dynamic_domains_mysql_redis_27 / mysql_redis / 69764e3
Reports root/ big
OK: 4736 / Failed: 0 / User-skipped: 154 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_27 / odbc_mssql_mnesia / 69764e3
Reports root/ big
OK: 4506 / Failed: 3 / User-skipped: 116 / Auto-skipped: 265

graphql_server_SUITE:admin_cli:clustering_tests:remove_alive_from_cluster
{error,{{badrpc,nodedown},
    [{distributed_helper,rpc,
               [#{timeout => 60000,
                node => mongooseim3@localhost},
                mongoose_cluster,join,
                [mongooseim@localhost]],
               [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                {line,140}]},
     {graphql_server_SUITE,remove_alive_from_cluster,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,214}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1303}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1235}]}]}}

Report log

graphql_server_SUITE:admin_cli:clustering_tests:remove_node_test
{error,{#{what => invalid_response_code,expected_type => ok,
      response_code => {exit_status,3}},
    [{graphql_helper,assert_response_code,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,258}]},
     {graphql_helper,get_ok_value,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,241}]},
     {graphql_server_SUITE,remove_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,225}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1303}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1235}]}]}}

Report log

graphql_server_SUITE:admin_cli:clustering_tests:stop_node_test
{error,{#{what => invalid_response_code,expected_type => ok,
      response_code => {exit_status,3}},
    [{graphql_helper,assert_response_code,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,258}]},
     {graphql_helper,get_ok_value,2,
             [{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
              {line,241}]},
     {graphql_server_SUITE,stop_node_test,1,
                 [{file,"/home/circleci/project/big_tests/tests/graphql_server_SUITE.erl"},
                {line,230}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1303}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1235}]}]}}

Report log

last_SUITE:init_per_suite
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

metrics_api_SUITE:init_per_suite
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

persistent_cluster_id_SUITE:init_per_suite
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

sasl_external_SUITE:init_per_suite
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

service_domain_db_SUITE:init_per_suite
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

service_mongoose_system_metrics_SUITE:init_per_suite
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log

shutdown_SUITE:init_per_suite
{fail,[{validate_node_failed,{badrpc,nodedown},mongooseim3@localhost}]}

Report log


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 69764e3
Reports root/ big
OK: 4771 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


internal_mnesia_27 / internal_mnesia / 69764e3
Reports root/ big
OK: 2492 / Failed: 0 / User-skipped: 770 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_27 / pgsql_mnesia / 69764e3
Reports root/ big
OK: 4771 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


pgsql_cets_27 / pgsql_cets / 69764e3
Reports root/ big
OK: 4847 / Failed: 0 / User-skipped: 188 / Auto-skipped: 0


mysql_redis_27 / mysql_redis / 69764e3
Reports root/ big
OK: 5127 / Failed: 0 / User-skipped: 149 / Auto-skipped: 0


pgsql_mnesia_26 / pgsql_mnesia / 69764e3
Reports root/ big
OK: 5148 / Failed: 0 / User-skipped: 128 / Auto-skipped: 0


pgsql_mnesia_27 / pgsql_mnesia / 69764e3
Reports root/ big
OK: 5148 / Failed: 0 / User-skipped: 128 / Auto-skipped: 0


cockroachdb_cets_27 / cockroachdb_cets / 69764e3
Reports root/ big
OK: 4843 / Failed: 1 / User-skipped: 188 / Auto-skipped: 3

pubsub_SUITE:tree+hometree_specific:can_create_node_with_existing_parent_path
{error,{{badmatch,false},
    [{pubsub_tools,check_response,2,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,444}]},
     {pubsub_tools,receive_response,3,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,434}]},
     {pubsub_tools,receive_and_check_response,4,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
            {line,424}]},
     {pubsub_SUITE,'-can_create_node_with_existing_parent_path/1-fun-0-',
             1,
             [{file,"/home/circleci/project/big_tests/tests/pubsub_SUITE.erl"},
            {line,1830}]},
     {escalus_story,story,4,
            [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"},
             {line,72}]},
     {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
     {test_server,run_test_case_eval1,6,
            [{file,"test_server.erl"},{line,1303}]},
     {test_server,run_test_case_eval,9,
            [{file,"test_server.erl"},{line,1235}]}]}}

Report log


mssql_mnesia_27 / odbc_mssql_mnesia / 69764e3
Reports root/ big
OK: 5143 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0


cockroachdb_cets_27 / cockroachdb_cets / 69764e3
Reports root/ big
OK: 117 / Failed: 0 / User-skipped: 2 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_27 / odbc_mssql_mnesia / 69764e3
Reports root/ big
OK: 225 / Failed: 3 / User-skipped: 36 / Auto-skipped: 0

@NelsonVides NelsonVides marked this pull request as ready for review January 8, 2025 08:05
Copy link
Member

@chrzaszcz chrzaszcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment about shapers.

doc/configuration/listen.md Outdated Show resolved Hide resolved
doc/configuration/listen.md Outdated Show resolved Hide resolved
@mongoose-im
Copy link
Collaborator

mongoose-im commented Jan 8, 2025

elasticsearch_and_cassandra_27 / elasticsearch_and_cassandra_mnesia / 4a99ce6
Reports root/ big
OK: 472 / Failed: 0 / User-skipped: 49 / Auto-skipped: 0


small_tests_27 / small_tests / 4a99ce6
Reports root / small


small_tests_26 / small_tests / 4a99ce6
Reports root / small


small_tests_27_arm64 / small_tests / 4a99ce6
Reports root / small


ldap_mnesia_26 / ldap_mnesia / 4a99ce6
Reports root/ big
OK: 2350 / Failed: 0 / User-skipped: 912 / Auto-skipped: 0


ldap_mnesia_27 / ldap_mnesia / 4a99ce6
Reports root/ big
OK: 2350 / Failed: 0 / User-skipped: 912 / Auto-skipped: 0


dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / 4a99ce6
Reports root/ big
OK: 4771 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


dynamic_domains_mysql_redis_27 / mysql_redis / 4a99ce6
Reports root/ big
OK: 4736 / Failed: 0 / User-skipped: 154 / Auto-skipped: 0


internal_mnesia_27 / internal_mnesia / 4a99ce6
Reports root/ big
OK: 2229 / Failed: 4 / User-skipped: 631 / Auto-skipped: 398

cluster_commands_SUITE:join_twice_in_parallel_using_rpc
{'EXIT',{{badrpc,timeout},
     {distributed_helper,rpc,
               [#{timeout => 60000,
                node => mongooseim2@localhost},
                mongoose_cluster,join,
                [mongooseim@localhost]],
               [{file,"/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
                {line,140}]}}}

Report log

cluster_commands_SUITE:cluster_of_three
{error,
  {{assertEqual,
     [{module,cluster_commands_SUITE},
      {line,357},
      {expression,"OpCode1"},
      {expected,0},
      {value,1}]},
   [{cluster_commands_SUITE,cluster_of_three,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cluster_commands_SUITE.erl"},
       {line,357}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1303}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1235}]}]}}

Report log

cluster_commands_SUITE:leave_the_three
{error,
  {{badrpc,timeout},
   [{distributed_helper,rpc,
      [#{timeout => 60000,node => mongooseim2@localhost},
       mongoose_cluster,join,
       [mongooseim@localhost]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {cluster_commands_SUITE,leave_the_three,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cluster_commands_SUITE.erl"},
       {line,369}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1303}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1235}]}]}}

Report log

cluster_commands_SUITE:remove_alive_from_cluster
{error,
  {{badrpc,timeout},
   [{distributed_helper,rpc,
      [#{timeout => 60000,node => mongooseim2@localhost},
       mongoose_cluster,join,
       [mongooseim@localhost]],
      [{file,
         "/home/circleci/project/big_tests/../test/common/distributed_helper.erl"},
       {line,140}]},
    {cluster_commands_SUITE,remove_alive_from_cluster,1,
      [{file,
         "/home/circleci/project/big_tests/tests/cluster_commands_SUITE.erl"},
       {line,411}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1794}]},
    {test_server,run_test_case_eval1,6,
      [{file,"test_server.erl"},{line,1303}]},
    {test_server,run_test_case_eval,9,
      [{file,"test_server.erl"},{line,1235}]}]}}

Report log

component_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

dynamic_domains_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

graphql_server_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

jingle_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

last_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

local_iq_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

metrics_api_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

mod_global_distrib_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

persistent_cluster_id_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

s2s_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

sasl_external_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

service_domain_db_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

service_mongoose_system_metrics_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

shutdown_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log

system_probes_SUITE:init_per_suite
{fail,[{validate_node_failed,mongooseim_not_running,mongooseim2@localhost}]}

Report log


dynamic_domains_pgsql_mnesia_27 / pgsql_mnesia / 4a99ce6
Reports root/ big
OK: 4771 / Failed: 0 / User-skipped: 119 / Auto-skipped: 0


dynamic_domains_mssql_mnesia_27 / odbc_mssql_mnesia / 4a99ce6
Reports root/ big
OK: 4766 / Failed: 0 / User-skipped: 124 / Auto-skipped: 0


pgsql_cets_27 / pgsql_cets / 4a99ce6
Reports root/ big
OK: 4847 / Failed: 0 / User-skipped: 188 / Auto-skipped: 0


pgsql_mnesia_27 / pgsql_mnesia / 4a99ce6
Reports root/ big
OK: 5148 / Failed: 0 / User-skipped: 128 / Auto-skipped: 0


mysql_redis_27 / mysql_redis / 4a99ce6
Reports root/ big
OK: 5127 / Failed: 0 / User-skipped: 149 / Auto-skipped: 0


cockroachdb_cets_27 / cockroachdb_cets / 4a99ce6
Reports root/ big
OK: 4871 / Failed: 2 / User-skipped: 188 / Auto-skipped: 0

pubsub_SUITE:dag+basic:request_particular_item_test
{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"alice_request_particular_item_test_2893@localhost/res1">>,
          escalus_tcp,<0.102325.0>,
          [{event_manager,<0.102239.0>},
           {server,<<"localhost">>},
           {username,<<"alicE_request_particular_item_test_2893">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.102239.0>},
            {server,<<"localhost">>},
            {username,
              <<"alicE_request_particular_item_test_2893">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,<<"alice_request_particular_item_test_2893">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,<<"alicE_request_particular_item_test_2893">>},
           {server,<<"localhost">>},
           {password,<<"matygrysa">>},
           {stream_id,<<"f60a136fc23731b5">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {pubsub_tools,receive_response,3,
       [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
        {line,433}]},
     {pubsub_tools,receive_and_check_response,4,
       [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
     ...

Report log

pubsub_SUITE:dag+manage_subscriptions:retrieve_node_subscriptions_test
{error,
  {timeout_when_waiting_for_stanza,
    [{escalus_client,wait_for_stanza,
       [{client,
          <<"alice_retrieve_node_subscriptions_test_2934@localhost/res1">>,
          escalus_tcp,<0.103349.0>,
          [{event_manager,<0.103317.0>},
           {server,<<"localhost">>},
           {username,
             <<"alicE_retrieve_node_subscriptions_test_2934">>},
           {resource,<<"res1">>}],
          [{event_client,
             [{event_manager,<0.103317.0>},
            {server,<<"localhost">>},
            {username,
              <<"alicE_retrieve_node_subscriptions_test_2934">>},
            {resource,<<"res1">>}]},
           {resource,<<"res1">>},
           {username,
             <<"alice_retrieve_node_subscriptions_test_2934">>},
           {server,<<"localhost">>},
           {host,<<"localhost">>},
           {port,5222},
           {auth,{escalus_auth,auth_plain}},
           {wspath,undefined},
           {username,
             <<"alicE_retrieve_node_subscriptions_test_2934">>},
           {server,<<"localhost">>},
           {password,<<"matygrysa">>},
           {stream_id,<<"3307524545cedb72">>}]},
        5000],
       [{file,
          "/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_client.erl"},
        {line,136}]},
     {pubsub_tools,receive_response,3,
       [{file,"/home/circleci/project/big_tests/tests/pubsub_tools.erl"},
        {line,433}]},
     {pubsub_tools,receive_and_check_response,4,
       [{file,"/h...

Report log


pgsql_mnesia_26 / pgsql_mnesia / 4a99ce6
Reports root/ big
OK: 5148 / Failed: 0 / User-skipped: 128 / Auto-skipped: 0


mssql_mnesia_27 / odbc_mssql_mnesia / 4a99ce6
Reports root/ big
OK: 5143 / Failed: 0 / User-skipped: 133 / Auto-skipped: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants